60]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_694(8157200) in day on r807u11n04 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_695(8157201) in day on r807u11n04 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_696(8157202) in day on r807u11n04 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_697(8157203) in day on r807u11n04 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_698(8157204) in day on r807u11n04 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_699(8157205) in day on r807u11n04 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3449(8157206) in gpu on r903u34n01 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3450(8157207) in gpu on r808u08n01 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3451(8157208) in gpu on r808u11n01 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3452(8157209) in gpu on r4513u23n01 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1716(8157210) in day on r909u04n02 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1717(8157211) in day on r909u10n01 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1718(8157212) in day on r909u10n03 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1719(8157213) in day on r918u05n02 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1720(8157214) in day on r919u05n04 Nov 04 20:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1721(8157215) in day on r919u32n04 Nov 04 20:37:58 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:234 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 20:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_983(7571949) WEXITSTATUS 0 Nov 04 20:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_983(7571949) done Nov 04 20:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_701(7572870) WEXITSTATUS 0 Nov 04 20:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_701(7572870) done Nov 04 20:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1145(7573314) WEXITSTATUS 0 Nov 04 20:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1145(7573314) done Nov 04 20:38:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7568129_1109(7569238) Nov 04 20:38:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7569662_1148(7570810) Nov 04 20:38:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_652(7571617) Nov 04 20:38:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_806(8127824) WEXITSTATUS 0 Nov 04 20:38:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_806(8127824) done Nov 04 20:38:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4755(8157216) in scavenge on r801u02n04 Nov 04 20:38:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4756(8157217) in scavenge on r802u25n04 Nov 04 20:38:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4757(8157218) in scavenge on r915u32n01 Nov 04 20:38:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4758(8157219) in scavenge on r808u11n01 Nov 04 20:38:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2181(8126036) WEXITSTATUS 1 Nov 04 20:38:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2181(8126036) done Nov 04 20:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3432(8156842) WEXITSTATUS 0 Nov 04 20:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3432(8156842) done Nov 04 20:38:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4220(8152275) has been requeued to reclaim resources for JobId=Invalid Nov 04 20:38:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4306(8154072) has been requeued to reclaim resources for JobId=Invalid Nov 04 20:38:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4317(8154200) has been requeued to reclaim resources for JobId=Invalid Nov 04 20:38:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4660(8156322) has been requeued to reclaim resources for JobId=Invalid Nov 04 20:38:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4220(8152275) Nov 04 20:38:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4317(8154200) Nov 04 20:38:39 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4660(8156322) Nov 04 20:38:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4306(8154072) Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:38:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:38:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4487(8155860) WEXITSTATUS 1 Nov 04 20:38:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4487(8155860) done Nov 04 20:38:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156259 WEXITSTATUS 0 Nov 04 20:38:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156259 done Nov 04 20:39:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3442(8157085) WEXITSTATUS 0 Nov 04 20:39:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3442(8157085) done Nov 04 20:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156038 NodeList=r816u23n07 #CPUs=12 Partition=scavenge Nov 04 20:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_836(8127915) WEXITSTATUS 0 Nov 04 20:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_836(8127915) done Nov 04 20:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4756(8157217) WEXITSTATUS 1 Nov 04 20:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4756(8157217) done Nov 04 20:39:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4341(8154951) WEXITSTATUS 1 Nov 04 20:39:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4341(8154951) done Nov 04 20:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1841(8151167) WEXITSTATUS 0 Nov 04 20:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1841(8151167) done Nov 04 20:39:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157220 InitPrio=57392 usec=4446 Nov 04 20:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4353(8154963) WEXITSTATUS 1 Nov 04 20:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4353(8154963) done Nov 04 20:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156264 WEXITSTATUS 1 Nov 04 20:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156264 done Nov 04 20:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4442(8155755) WEXITSTATUS 1 Nov 04 20:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4442(8155755) done Nov 04 20:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1576(8150555) WEXITSTATUS 0 Nov 04 20:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1576(8150555) done Nov 04 20:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4540(8155945) WEXITSTATUS 1 Nov 04 20:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4540(8155945) done Nov 04 20:40:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3450(8157207) WEXITSTATUS 0 Nov 04 20:40:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3450(8157207) done Nov 04 20:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1972(8151410) WEXITSTATUS 0 Nov 04 20:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1972(8151410) done Nov 04 20:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3406(8156235) WEXITSTATUS 0 Nov 04 20:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3406(8156235) done Nov 04 20:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3808(8144422) WEXITSTATUS 1 Nov 04 20:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3808(8144422) done Nov 04 20:40:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1332(8150064) WEXITSTATUS 0 Nov 04 20:40:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1332(8150064) done Nov 04 20:40:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3444(8157087) WEXITSTATUS 0 Nov 04 20:40:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3444(8157087) done Nov 04 20:40:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157221 InitPrio=27676 usec=8581 Nov 04 20:40:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157221 NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 04 20:40:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4453(8155766) WEXITSTATUS 1 Nov 04 20:40:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4453(8155766) done Nov 04 20:40:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_916(8128123) WEXITSTATUS 0 Nov 04 20:40:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_916(8128123) done Nov 04 20:40:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3561(8136017) WEXITSTATUS 1 Nov 04 20:40:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3561(8136017) done Nov 04 20:40:36 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_34(7570998) Nov 04 20:40:36 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146378 Nov 04 20:40:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157220 NodeList=r915u38n02 #CPUs=1 Partition=pi_lora Nov 04 20:40:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1988(8151426) WEXITSTATUS 0 Nov 04 20:40:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1988(8151426) done Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:40:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2087(8151763) WEXITSTATUS 0 Nov 04 20:40:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2087(8151763) done Nov 04 20:41:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4664(8156326) WEXITSTATUS 1 Nov 04 20:41:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4664(8156326) done Nov 04 20:41:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4548(8155957) WEXITSTATUS 1 Nov 04 20:41:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4548(8155957) done Nov 04 20:41:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3447(8157090) WEXITSTATUS 0 Nov 04 20:41:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3447(8157090) done Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_700(8157222) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_701(8157223) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_702(8157224) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_703(8157225) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_704(8157226) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_705(8157227) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_706(8157228) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_707(8157229) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_708(8157230) in day on r805u04n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_709(8157231) in day on r805u11n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_710(8157232) in day on r805u11n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_711(8157233) in day on r805u11n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_712(8157234) in day on r913u01n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_713(8157235) in day on r913u01n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_714(8157236) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_715(8157237) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_716(8157238) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_717(8157239) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_718(8157240) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_719(8157241) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_720(8157242) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_721(8157243) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_722(8157244) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_723(8157245) in day on r913u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_724(8157246) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_725(8157247) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_726(8157248) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_727(8157249) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_728(8157250) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_729(8157251) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_730(8157252) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_731(8157253) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_732(8157254) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_733(8157255) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_734(8157256) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_735(8157257) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_736(8157258) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_737(8157259) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_738(8157260) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_739(8157261) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_740(8157262) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_741(8157263) in day on r806u02n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_742(8157264) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_743(8157265) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_744(8157266) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_745(8157267) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_746(8157268) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_747(8157269) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_748(8157270) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_749(8157271) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_750(8157272) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_751(8157273) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_752(8157274) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_753(8157275) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_754(8157276) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_755(8157277) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_756(8157278) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_757(8157279) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_758(8157280) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_759(8157281) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_760(8157282) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_761(8157283) in day on r806u02n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_762(8157284) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_763(8157285) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_764(8157286) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_765(8157287) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_766(8157288) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_767(8157289) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_768(8157290) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_769(8157291) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_770(8157292) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_771(8157293) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_772(8157294) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_773(8157295) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_774(8157296) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_775(8157297) in day on r806u06n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_776(8157298) in day on r806u06n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_777(8157299) in day on r806u06n03 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_778(8157300) in day on r807u03n01 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_779(8157301) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_780(8157302) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_781(8157303) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_782(8157304) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_783(8157305) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_784(8157306) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_785(8157307) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_786(8157308) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_787(8157309) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_788(8157310) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_789(8157311) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_790(8157312) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_791(8157313) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_792(8157314) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_793(8157315) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_794(8157316) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_795(8157317) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_796(8157318) in day on r807u05n02 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_797(8157319) in day on r807u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_798(8157320) in day on r807u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_799(8157321) in day on r807u05n04 Nov 04 20:41:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3453(8157322) in gpu on r903u34n01 Nov 04 20:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3454(8157323) in gpu on r808u08n01 Nov 04 20:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3455(8157324) in gpu on r808u08n01 Nov 04 20:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3456(8157325) in gpu on r808u14n01 Nov 04 20:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3457(8157326) in gpu on r808u14n01 Nov 04 20:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3458(8157327) in gpu on r808u17n01 Nov 04 20:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1722(8157328) in day on r918u09n04 Nov 04 20:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1723(8157329) in day on r918u36n03 Nov 04 20:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u05n04 Nov 04 20:41:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4074(8148133) WEXITSTATUS 1 Nov 04 20:41:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4074(8148133) done Nov 04 20:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4689(8156461) WEXITSTATUS 1 Nov 04 20:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4689(8156461) done Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4759(8157330) in scavenge on r801u27n02 Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4760(8157331) in scavenge on r802u25n01 Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4761(8157332) in scavenge on r915u35n04 Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4762(8157333) in scavenge on r815u03n12 Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4763(8157334) in scavenge on r815u15n12 Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4764(8157335) in scavenge on r919u09n03 Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4765(8157336) in scavenge on r815u29n01 Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4766(8157337) in scavenge on r816u03n04 Nov 04 20:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8149759_11(8157338) in scavenge on r917u07n02 Nov 04 20:41:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4052(8147293) WEXITSTATUS 1 Nov 04 20:41:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4052(8147293) done Nov 04 20:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1567(8150546) WEXITSTATUS 0 Nov 04 20:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1567(8150546) done Nov 04 20:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3457(8157326) WEXITSTATUS 0 Nov 04 20:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3457(8157326) done Nov 04 20:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4444(8155757) WEXITSTATUS 1 Nov 04 20:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4444(8155757) done Nov 04 20:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1984(8151422) WEXITSTATUS 0 Nov 04 20:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1984(8151422) done Nov 04 20:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3455(8157324) WEXITSTATUS 0 Nov 04 20:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3455(8157324) done Nov 04 20:41:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3849(8144678) WEXITSTATUS 1 Nov 04 20:41:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3849(8144678) done Nov 04 20:41:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3962(8146005) WEXITSTATUS 1 Nov 04 20:41:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3962(8146005) done Nov 04 20:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_833(8127912) WEXITSTATUS 0 Nov 04 20:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_833(8127912) done Nov 04 20:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1437(8135970) in scavenge on r907u28n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:41:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:41:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1645(8150735) WEXITSTATUS 0 Nov 04 20:41:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1645(8150735) done Nov 04 20:41:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157339 InitPrio=57392 usec=4710 Nov 04 20:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_519(7571484) WEXITSTATUS 0 Nov 04 20:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_519(7571484) done Nov 04 20:42:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8138031 Nov 04 20:42:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8138031 error Job/step already completing or completed Nov 04 20:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1094(8131103) WEXITSTATUS 0 Nov 04 20:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1094(8131103) done Nov 04 20:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4519(8155908) WEXITSTATUS 1 Nov 04 20:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4519(8155908) done Nov 04 20:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_33(8135732) WEXITSTATUS 0 Nov 04 20:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_33(8135732) done Nov 04 20:42:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1562(8150541) WEXITSTATUS 0 Nov 04 20:42:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1562(8150541) done Nov 04 20:42:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4753(8157103) WEXITSTATUS 1 Nov 04 20:42:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4753(8157103) done Nov 04 20:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3454(8157323) WEXITSTATUS 0 Nov 04 20:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3454(8157323) done Nov 04 20:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3458(8157327) WEXITSTATUS 0 Nov 04 20:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3458(8157327) done Nov 04 20:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156233 WEXITSTATUS 0 Nov 04 20:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156233 done Nov 04 20:42:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 20:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3449(8157206) WEXITSTATUS 0 Nov 04 20:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3449(8157206) done Nov 04 20:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4726(8156855) WEXITSTATUS 1 Nov 04 20:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4726(8156855) done Nov 04 20:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4257(8153133) WEXITSTATUS 1 Nov 04 20:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4257(8153133) done Nov 04 20:42:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157340 InitPrio=77209 usec=3460 Nov 04 20:42:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_773(8127712) WEXITSTATUS 0 Nov 04 20:42:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_773(8127712) done Nov 04 20:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157340 NodeList=r802u14n03 #CPUs=1 Partition=pi_manohar Nov 04 20:43:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3949(8145886) WEXITSTATUS 1 Nov 04 20:43:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3949(8145886) done Nov 04 20:43:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_11(8135710) WEXITSTATUS 0 Nov 04 20:43:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_11(8135710) done Nov 04 20:43:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8157341 NodeList=(null) usec=23108 Nov 04 20:43:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157341 NodeList=r815u15n11 #CPUs=8 Partition=pi_polimanti Nov 04 20:43:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157339 NodeList=r912u13n01 #CPUs=1 Partition=bigmem Nov 04 20:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4375(8155202) WEXITSTATUS 1 Nov 04 20:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4375(8155202) done Nov 04 20:43:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_867(7571833) Nov 04 20:43:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_908(7571874) Nov 04 20:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1676(8150766) WEXITSTATUS 0 Nov 04 20:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1676(8150766) done Nov 04 20:43:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4743(8156983) WEXITSTATUS 1 Nov 04 20:43:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4743(8156983) done Nov 04 20:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_850(8127953) WEXITSTATUS 0 Nov 04 20:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_850(8127953) done Nov 04 20:43:37 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8155969 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:43:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1007(8128445) WEXITSTATUS 0 Nov 04 20:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1007(8128445) done Nov 04 20:44:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156039 NodeList=r815u23n08 #CPUs=12 Partition=scavenge Nov 04 20:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2070(8151630) WEXITSTATUS 0 Nov 04 20:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2070(8151630) done Nov 04 20:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2493(8127552) WEXITSTATUS 1 Nov 04 20:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2493(8127552) done Nov 04 20:44:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3935(8145659) WEXITSTATUS 1 Nov 04 20:44:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3935(8145659) done Nov 04 20:44:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157342 InitPrio=20041 usec=3994 Nov 04 20:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155968 WEXITSTATUS 0 Nov 04 20:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155968 done Nov 04 20:44:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4503(8155886) WEXITSTATUS 1 Nov 04 20:44:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4503(8155886) done Nov 04 20:44:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_800(8157343) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_801(8157344) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_802(8157345) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_803(8157346) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_804(8157347) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_805(8157348) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_806(8157349) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_807(8157350) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_808(8157351) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_809(8157352) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_810(8157353) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_811(8157354) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_812(8157355) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_813(8157356) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_814(8157357) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_815(8157358) in day on r805u06n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_816(8157359) in day on r805u09n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_817(8157360) in day on r805u09n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_818(8157361) in day on r806u02n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_819(8157362) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_820(8157363) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_821(8157364) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_822(8157365) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_823(8157366) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_824(8157367) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_825(8157368) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_826(8157369) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_827(8157370) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_828(8157371) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_829(8157372) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_830(8157373) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_831(8157374) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_832(8157375) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_833(8157376) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_834(8157377) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_835(8157378) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_836(8157379) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_837(8157380) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_838(8157381) in day on r806u04n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_839(8157382) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_840(8157383) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_841(8157384) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_842(8157385) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_843(8157386) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_844(8157387) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_845(8157388) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_846(8157389) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_847(8157390) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_848(8157391) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_849(8157392) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_850(8157393) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_851(8157394) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_852(8157395) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_853(8157396) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_854(8157397) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_855(8157398) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_856(8157399) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_857(8157400) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_858(8157401) in day on r912u28n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_859(8157402) in day on r913u13n02 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_860(8157403) in day on r913u28n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_861(8157404) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_862(8157405) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_863(8157406) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_864(8157407) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_865(8157408) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_866(8157409) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_867(8157410) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_868(8157411) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_869(8157412) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_870(8157413) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_871(8157414) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_872(8157415) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_873(8157416) in day on r807u05n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_874(8157417) in day on r807u08n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_875(8157418) in day on r807u08n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_876(8157419) in day on r807u08n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_877(8157420) in day on r807u11n03 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_878(8157421) in day on r807u11n03 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_879(8157422) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_880(8157423) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_881(8157424) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_882(8157425) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_883(8157426) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_884(8157427) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_885(8157428) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_886(8157429) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_887(8157430) in day on r807u11n04 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_888(8157431) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_889(8157432) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_890(8157433) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_891(8157434) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_892(8157435) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_893(8157436) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_894(8157437) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_895(8157438) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_896(8157439) in day on r909u01n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_897(8157440) in day on r909u07n03 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_898(8157441) in day on r909u07n03 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_899(8157442) in day on r909u07n03 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157342 in day on r815u03n[02-03,05-06],r815u09n[02,05,07],r815u15n[02-05],r815u23n05 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3459(8157443) in gpu on r903u34n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3460(8157444) in gpu on r808u08n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3461(8157445) in gpu on r808u08n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3462(8157446) in gpu on r808u14n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3463(8157447) in gpu on r808u17n01 Nov 04 20:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1724(8157448) in day on r816u03n01 Nov 04 20:44:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156040 in scavenge on r815u15n11 Nov 04 20:44:29 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:222 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 20:44:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3446(8157089) WEXITSTATUS 0 Nov 04 20:44:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3446(8157089) done Nov 04 20:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1179(7572146) WEXITSTATUS 0 Nov 04 20:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1179(7572146) done Nov 04 20:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_140(7572307) WEXITSTATUS 0 Nov 04 20:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_140(7572307) done Nov 04 20:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4396(8155326) WEXITSTATUS 1 Nov 04 20:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4396(8155326) done Nov 04 20:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_383(7572551) WEXITSTATUS 0 Nov 04 20:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_383(7572551) done Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4767(8157449) in scavenge on r907u05n01 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4768(8157450) in scavenge on r907u09n02 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4769(8157451) in scavenge on r903u04n03 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4770(8157452) in scavenge on r916u04n02 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4771(8157453) in scavenge on r802u06n04 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4772(8157454) in scavenge on r802u11n02 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4773(8157455) in scavenge on r802u25n01 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4774(8157456) in scavenge on r807u22n01 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4775(8157457) in scavenge on r919u01n01 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4776(8157458) in scavenge on r919u09n01 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4777(8157459) in scavenge on r919u09n04 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4778(8157460) in scavenge on r815u15n06 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4779(8157461) in scavenge on r816u23n05 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4780(8157462) in scavenge on r815u23n02 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4781(8157463) in scavenge on r815u29n05 Nov 04 20:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8149759_12(8157464) in scavenge on r802u25n03 Nov 04 20:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3462(8157446) WEXITSTATUS 0 Nov 04 20:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3462(8157446) done Nov 04 20:44:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1172(8133635) WEXITSTATUS 0 Nov 04 20:44:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1172(8133635) done Nov 04 20:44:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157465 InitPrio=13461 usec=5537 Nov 04 20:44:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4745(8157095) WEXITSTATUS 1 Nov 04 20:44:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4745(8157095) done Nov 04 20:44:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4414(8155553) WEXITSTATUS 1 Nov 04 20:44:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4414(8155553) done Nov 04 20:44:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3459(8157443) WEXITSTATUS 0 Nov 04 20:44:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3459(8157443) done Nov 04 20:45:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4474(8155791) WEXITSTATUS 1 Nov 04 20:45:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4474(8155791) done Nov 04 20:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1171(8133634) WEXITSTATUS 0 Nov 04 20:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1171(8133634) done Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:45:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1580(8150559) WEXITSTATUS 0 Nov 04 20:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1580(8150559) done Nov 04 20:45:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4645(8156249) WEXITSTATUS 1 Nov 04 20:45:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4645(8156249) done Nov 04 20:45:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2152(8151828) WEXITSTATUS 0 Nov 04 20:45:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2152(8151828) done Nov 04 20:45:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1191(7572158) Nov 04 20:45:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3760(8142738) WEXITSTATUS 1 Nov 04 20:45:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3760(8142738) done Nov 04 20:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1805(8151016) WEXITSTATUS 0 Nov 04 20:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1805(8151016) done Nov 04 20:45:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1744(8150955) WEXITSTATUS 0 Nov 04 20:45:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1744(8150955) done Nov 04 20:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1890(8151216) WEXITSTATUS 0 Nov 04 20:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1890(8151216) done Nov 04 20:46:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156041 NodeList=r816u29n07 #CPUs=12 Partition=scavenge Nov 04 20:46:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4394(8155324) WEXITSTATUS 1 Nov 04 20:46:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4394(8155324) done Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157466 InitPrio=32151 usec=1937 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157467 InitPrio=32151 usec=2432 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157468 InitPrio=32151 usec=1606 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157469 InitPrio=32151 usec=1388 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157470 InitPrio=32151 usec=1284 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157471 InitPrio=32151 usec=2504 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157472 InitPrio=32151 usec=1814 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157473 InitPrio=32151 usec=2507 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157474 InitPrio=32151 usec=1762 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157475 InitPrio=32151 usec=1269 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157476 InitPrio=32151 usec=1199 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157477 InitPrio=32151 usec=1129 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157478 InitPrio=32151 usec=1836 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157479 InitPrio=32151 usec=1218 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157480 InitPrio=32151 usec=1456 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157481 InitPrio=32151 usec=1267 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157482 InitPrio=32151 usec=1158 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157483 InitPrio=32151 usec=1395 Nov 04 20:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157484 InitPrio=32151 usec=1186 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157485 InitPrio=32151 usec=1880 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157486 InitPrio=32151 usec=1539 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157487 InitPrio=32151 usec=1343 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157488 InitPrio=32151 usec=1109 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157489 InitPrio=32151 usec=1081 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157490 InitPrio=32151 usec=1124 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157491 InitPrio=32151 usec=1033 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157492 InitPrio=32151 usec=1026 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157493 InitPrio=32151 usec=1058 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157494 InitPrio=32151 usec=2220 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157495 InitPrio=32151 usec=1614 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157496 InitPrio=32151 usec=2015 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157497 InitPrio=32151 usec=1528 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157498 InitPrio=32151 usec=2067 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157499 InitPrio=32151 usec=1486 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157500 InitPrio=32151 usec=1337 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157501 InitPrio=32151 usec=1319 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157502 InitPrio=32151 usec=1193 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157503 InitPrio=32151 usec=1164 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157504 InitPrio=32151 usec=1173 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157505 InitPrio=32151 usec=1148 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157506 InitPrio=32151 usec=1580 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157507 InitPrio=32151 usec=1360 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157508 InitPrio=32151 usec=1160 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157509 InitPrio=32151 usec=1094 Nov 04 20:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157510 InitPrio=32151 usec=1160 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157511 InitPrio=32151 usec=1086 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157512 InitPrio=32151 usec=1292 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157513 InitPrio=32151 usec=1143 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157514 InitPrio=32151 usec=1178 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157515 InitPrio=32151 usec=1046 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157516 InitPrio=32151 usec=2306 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157517 InitPrio=32151 usec=1595 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157518 InitPrio=32151 usec=1990 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157519 InitPrio=32151 usec=1647 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157520 InitPrio=32151 usec=1311 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157521 InitPrio=32151 usec=1323 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157522 InitPrio=32151 usec=1274 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157523 InitPrio=32151 usec=1113 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157524 InitPrio=32151 usec=1093 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157525 InitPrio=32151 usec=1105 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157526 InitPrio=32151 usec=1632 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157527 InitPrio=32151 usec=1333 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157528 InitPrio=32151 usec=1140 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157529 InitPrio=32151 usec=1279 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157530 InitPrio=32151 usec=1156 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157531 InitPrio=32151 usec=1041 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157532 InitPrio=32151 usec=1314 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157533 InitPrio=32151 usec=1279 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157534 InitPrio=32151 usec=1050 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157535 InitPrio=32151 usec=1104 Nov 04 20:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157536 InitPrio=32151 usec=1137 Nov 04 20:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157537 InitPrio=32151 usec=2303 Nov 04 20:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157538 InitPrio=32151 usec=1932 Nov 04 20:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157539 InitPrio=32151 usec=1823 Nov 04 20:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157540 InitPrio=32151 usec=1419 Nov 04 20:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157541 InitPrio=32151 usec=1254 Nov 04 20:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157542 InitPrio=32151 usec=1819 Nov 04 20:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157543 InitPrio=32151 usec=1315 Nov 04 20:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157544 InitPrio=32151 usec=1155 Nov 04 20:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157545 InitPrio=32151 usec=2336 Nov 04 20:46:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1641(8150731) WEXITSTATUS 0 Nov 04 20:46:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1641(8150731) done Nov 04 20:46:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4593(8156114) WEXITSTATUS 1 Nov 04 20:46:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4593(8156114) done Nov 04 20:46:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1632(8150722) WEXITSTATUS 0 Nov 04 20:46:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1632(8150722) done Nov 04 20:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1255(8149745) WEXITSTATUS 0 Nov 04 20:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1255(8149745) done Nov 04 20:46:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157546 InitPrio=32151 usec=1999 Nov 04 20:47:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1685(8150775) WEXITSTATUS 0 Nov 04 20:47:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1685(8150775) done Nov 04 20:47:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1568(8150547) WEXITSTATUS 0 Nov 04 20:47:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1568(8150547) done Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:47:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:47:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157547 InitPrio=32151 usec=1693 Nov 04 20:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1189(8133863) WEXITSTATUS 0 Nov 04 20:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1189(8133863) done Nov 04 20:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1563(8150542) WEXITSTATUS 0 Nov 04 20:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1563(8150542) done Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157465_0(8157548) in pi_balou on r911u09n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157465_1(8157549) in pi_balou on r911u09n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157465_2(8157550) in pi_balou on r911u09n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157465_3(8157551) in pi_balou on r911u09n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157465_4(8157552) in pi_balou on r911u09n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157465_5(8157553) in pi_balou on r911u13n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157465_6(8157465) in pi_balou on r911u13n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_900(8157554) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_901(8157555) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_902(8157556) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_903(8157557) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_904(8157558) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_905(8157559) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_906(8157560) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_907(8157561) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_908(8157562) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_909(8157563) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_910(8157564) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_911(8157565) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_912(8157566) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_913(8157567) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_914(8157568) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_915(8157569) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_916(8157570) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_917(8157571) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_918(8157572) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_919(8157573) in day on r807u05n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_920(8157574) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_921(8157575) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_922(8157576) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_923(8157577) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_924(8157578) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_925(8157579) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_926(8157580) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_927(8157581) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_928(8157582) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_929(8157583) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_930(8157584) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_931(8157585) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_932(8157586) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_933(8157587) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_934(8157588) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_935(8157589) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_936(8157590) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_937(8157591) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_938(8157592) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_939(8157593) in day on r904u04n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_940(8157594) in day on r805u02n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_941(8157595) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_942(8157596) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_943(8157597) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_944(8157598) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_945(8157599) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_946(8157600) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_947(8157601) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_948(8157602) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_949(8157603) in day on r805u02n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_950(8157604) in day on r806u04n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_951(8157605) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_952(8157606) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_953(8157607) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_954(8157608) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_955(8157609) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_956(8157610) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_957(8157611) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_958(8157612) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_959(8157613) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_960(8157614) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_961(8157615) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_962(8157616) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_963(8157617) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_964(8157618) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_965(8157619) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_966(8157620) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_967(8157621) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_968(8157622) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_969(8157623) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_970(8157624) in day on r806u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_971(8157625) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_972(8157626) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_973(8157627) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_974(8157628) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_975(8157629) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_976(8157630) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_977(8157631) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_978(8157632) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_979(8157633) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_980(8157634) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_981(8157635) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_982(8157636) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_983(8157637) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_984(8157638) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_985(8157639) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_986(8157640) in day on r912u28n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_987(8157641) in day on r912u28n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_988(8157642) in day on r912u28n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_989(8157643) in day on r912u28n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_990(8157644) in day on r912u36n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_991(8157645) in day on r912u36n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_992(8157646) in day on r912u36n02 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_993(8157647) in day on r913u05n03 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_994(8157648) in day on r913u13n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_995(8157649) in day on r913u28n04 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_996(8157650) in day on r909u01n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_997(8157651) in day on r909u01n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_998(8157652) in day on r909u01n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156351_999(8156351) in day on r909u01n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3464(8157653) in gpu on r903u34n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3465(8157654) in gpu on r808u14n01 Nov 04 20:47:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3466(8157655) in gpu on r808u17n01 Nov 04 20:47:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r909u01n01 Nov 04 20:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_388(7572556) WEXITSTATUS 0 Nov 04 20:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_388(7572556) done Nov 04 20:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_67(7572234) WEXITSTATUS 0 Nov 04 20:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_67(7572234) done Nov 04 20:47:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157656 InitPrio=32151 usec=2110 Nov 04 20:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_632(7572801) WEXITSTATUS 0 Nov 04 20:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_632(7572801) done Nov 04 20:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_634(7572803) WEXITSTATUS 0 Nov 04 20:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_634(7572803) done Nov 04 20:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1078(8130755) WEXITSTATUS 0 Nov 04 20:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1078(8130755) done Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4782(8157657) in scavenge on r912u05n04 Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4783(8157658) in scavenge on r916u01n01 Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4784(8157659) in scavenge on r917u29n02 Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4785(8157660) in scavenge on r903u34n01 Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4786(8157661) in scavenge on r919u01n01 Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4787(8157662) in scavenge on r909u13n03 Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4788(8157663) in scavenge on r909u13n03 Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4789(8157664) in scavenge on r909u29n02 Nov 04 20:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4790(8157665) in scavenge on r909u29n02 Nov 04 20:47:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_940(8128210) WEXITSTATUS 0 Nov 04 20:47:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_940(8128210) done Nov 04 20:48:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1952(8151390) WEXITSTATUS 0 Nov 04 20:48:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1952(8151390) done Nov 04 20:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2044(8151604) WEXITSTATUS 0 Nov 04 20:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2044(8151604) done Nov 04 20:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_997(8128386) WEXITSTATUS 0 Nov 04 20:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_997(8128386) done Nov 04 20:48:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155870 WEXITSTATUS 1 Nov 04 20:48:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155870 done Nov 04 20:48:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7569662_651(7570313) Nov 04 20:48:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_5(7570968) Nov 04 20:48:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_395(7572563) Nov 04 20:48:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3463(8157447) WEXITSTATUS 0 Nov 04 20:48:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3463(8157447) done Nov 04 20:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4787(8157662) WEXITSTATUS 1 Nov 04 20:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4787(8157662) done Nov 04 20:48:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157666 InitPrio=32151 usec=2089 Nov 04 20:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3167(8131315) WEXITSTATUS 0 Nov 04 20:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3167(8131315) done Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:48:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1185(8133859) WEXITSTATUS 0 Nov 04 20:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1185(8133859) done Nov 04 20:48:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1520(8150365) WEXITSTATUS 0 Nov 04 20:48:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1520(8150365) done Nov 04 20:48:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_822(8127865) WEXITSTATUS 0 Nov 04 20:48:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_822(8127865) done Nov 04 20:48:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1209(8134440) WEXITSTATUS 0 Nov 04 20:48:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1209(8134440) done Nov 04 20:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1229(8135095) WEXITSTATUS 0 Nov 04 20:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1229(8135095) done Nov 04 20:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_973(8128341) WEXITSTATUS 0 Nov 04 20:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_973(8128341) done Nov 04 20:48:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157667 InitPrio=32151 usec=2037 Nov 04 20:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3445(8157088) WEXITSTATUS 0 Nov 04 20:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3445(8157088) done Nov 04 20:48:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4149(8150165) WEXITSTATUS 1 Nov 04 20:48:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4149(8150165) done Nov 04 20:49:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2016(8151576) WEXITSTATUS 0 Nov 04 20:49:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2016(8151576) done Nov 04 20:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156042 NodeList=r816u15n11 #CPUs=12 Partition=scavenge Nov 04 20:49:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 20:49:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157668 InitPrio=32151 usec=1693 Nov 04 20:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1202(8134326) WEXITSTATUS 0 Nov 04 20:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1202(8134326) done Nov 04 20:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_835(8127914) WEXITSTATUS 0 Nov 04 20:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_835(8127914) done Nov 04 20:49:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157669 InitPrio=32151 usec=1818 Nov 04 20:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4648(8156252) WEXITSTATUS 1 Nov 04 20:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4648(8156252) done Nov 04 20:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4531(8155932) WEXITSTATUS 1 Nov 04 20:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4531(8155932) done Nov 04 20:50:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157670 InitPrio=32151 usec=2320 Nov 04 20:50:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4430(8155671) WEXITSTATUS 1 Nov 04 20:50:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4430(8155671) done Nov 04 20:50:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4429(8155670) WEXITSTATUS 1 Nov 04 20:50:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4429(8155670) done Nov 04 20:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4499(8155882) WEXITSTATUS 1 Nov 04 20:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4499(8155882) done Nov 04 20:50:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4764(8157335) WEXITSTATUS 1 Nov 04 20:50:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4764(8157335) done Nov 04 20:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1003(8128422) WEXITSTATUS 0 Nov 04 20:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1003(8128422) done Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:50:20 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:25 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:25 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:26 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 20:50:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4729(8156858) WEXITSTATUS 1 Nov 04 20:50:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4729(8156858) done Nov 04 20:50:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1638(8150728) WEXITSTATUS 0 Nov 04 20:50:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1638(8150728) done Nov 04 20:50:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_523(7571488) Nov 04 20:50:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1599(8124143) Nov 04 20:50:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1601(8124145) Nov 04 20:50:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157671 InitPrio=32151 usec=1662 Nov 04 20:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1104(8131596) WEXITSTATUS 0 Nov 04 20:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1104(8131596) done Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3467(8157672) in gpu on r903u34n01 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3468(8157673) in gpu on r808u17n01 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3469(8157674) in gpu on r808u17n01 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1725(8157675) in day on r805u09n02 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1726(8157676) in day on r904u04n02 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1727(8157677) in day on r913u24n04 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1728(8157678) in day on r806u04n04 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1729(8157679) in day on r806u14n01 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1730(8157680) in day on r909u04n03 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1731(8157681) in day on r909u13n02 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1732(8157682) in day on r918u01n01 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1733(8157683) in day on r919u01n01 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1734(8157684) in day on r919u05n02 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1735(8157685) in day on r919u05n03 Nov 04 20:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1736(8157686) in day on r919u32n03 Nov 04 20:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156043 in scavenge on r815u29n05 Nov 04 20:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1032(7571998) WEXITSTATUS 0 Nov 04 20:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1032(7571998) done Nov 04 20:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_588(7572757) WEXITSTATUS 0 Nov 04 20:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_588(7572757) done Nov 04 20:51:00 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4791(8157687) in scavenge on r801u06n02 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4792(8157688) in scavenge on r915u32n01 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4793(8157689) in scavenge on r917u07n04 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4794(8157690) in scavenge on r917u10n04 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4795(8157691) in scavenge on r816u35n07 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4796(8157692) in scavenge on r909u10n03 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4797(8157693) in scavenge on r918u01n01 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4798(8157694) in scavenge on r918u01n01 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4799(8157695) in scavenge on r918u01n03 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4800(8157696) in scavenge on r919u05n04 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4801(8157697) in scavenge on r919u09n03 Nov 04 20:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8149759_13(8157698) in scavenge on r903u04n02 Nov 04 20:51:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3468(8157673) WEXITSTATUS 0 Nov 04 20:51:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3468(8157673) done Nov 04 20:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2164(8151956) WEXITSTATUS 0 Nov 04 20:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2164(8151956) done Nov 04 20:51:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157699 InitPrio=32151 usec=1923 Nov 04 20:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8141906 WEXITSTATUS 0 Nov 04 20:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8141906 done Nov 04 20:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4566(8156023) WEXITSTATUS 1 Nov 04 20:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4566(8156023) done Nov 04 20:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1436(8135969) in scavenge on r801u06n02 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157700 InitPrio=32151 usec=2158 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157701 InitPrio=32151 usec=1983 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157702 InitPrio=32151 usec=1399 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157703 InitPrio=32151 usec=1145 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157704 InitPrio=32151 usec=1288 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157705 InitPrio=32151 usec=1296 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157706 InitPrio=32151 usec=1118 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157707 InitPrio=32151 usec=1084 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157708 InitPrio=32151 usec=1355 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157709 InitPrio=32151 usec=1035 Nov 04 20:51:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157710 InitPrio=32151 usec=979 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157711 InitPrio=32151 usec=2329 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157712 InitPrio=32151 usec=1259 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157713 InitPrio=32151 usec=1072 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157714 InitPrio=32151 usec=985 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157715 InitPrio=32151 usec=914 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157716 InitPrio=32151 usec=959 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157717 InitPrio=32151 usec=961 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157718 InitPrio=32151 usec=1358 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157719 InitPrio=32151 usec=2499 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157720 InitPrio=32151 usec=1736 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157721 InitPrio=32151 usec=1285 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157722 InitPrio=32151 usec=1328 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157723 InitPrio=32151 usec=1075 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157724 InitPrio=32151 usec=999 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157725 InitPrio=32151 usec=1059 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157726 InitPrio=32151 usec=1113 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157727 InitPrio=32151 usec=976 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157728 InitPrio=32151 usec=1586 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157729 InitPrio=32151 usec=1261 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157730 InitPrio=32151 usec=1056 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157731 InitPrio=32151 usec=971 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157732 InitPrio=32151 usec=953 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157733 InitPrio=32151 usec=918 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157734 InitPrio=32151 usec=938 Nov 04 20:51:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157735 InitPrio=32151 usec=1990 Nov 04 20:51:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157736 InitPrio=32151 usec=1282 Nov 04 20:51:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3056(8127787) WEXITSTATUS 0 Nov 04 20:51:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3056(8127787) done Nov 04 20:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1908(8151234) WEXITSTATUS 0 Nov 04 20:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1908(8151234) done Nov 04 20:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133117 WEXITSTATUS 0 Nov 04 20:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133117 done Nov 04 20:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1234(8135117) WEXITSTATUS 0 Nov 04 20:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1234(8135117) done Nov 04 20:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4433(8155674) WEXITSTATUS 1 Nov 04 20:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4433(8155674) done Nov 04 20:51:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4517(8155906) WEXITSTATUS 1 Nov 04 20:51:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4517(8155906) done Nov 04 20:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4252(8153128) WEXITSTATUS 1 Nov 04 20:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4252(8153128) done Nov 04 20:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1190(8133864) WEXITSTATUS 0 Nov 04 20:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1190(8133864) done Nov 04 20:52:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3883(8144948) WEXITSTATUS 1 Nov 04 20:52:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3883(8144948) done Nov 04 20:52:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1241(8135134) WEXITSTATUS 0 Nov 04 20:52:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1241(8135134) done Nov 04 20:52:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4452(8155765) WEXITSTATUS 1 Nov 04 20:52:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4452(8155765) done Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:52:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:52:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1222(8134911) WEXITSTATUS 0 Nov 04 20:52:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1222(8134911) done Nov 04 20:53:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156475 WEXITSTATUS 0 Nov 04 20:53:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156475 done Nov 04 20:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2054(8151614) WEXITSTATUS 0 Nov 04 20:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2054(8151614) done Nov 04 20:53:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1246(8135148) WEXITSTATUS 0 Nov 04 20:53:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1246(8135148) done Nov 04 20:53:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_775(8127720) WEXITSTATUS 0 Nov 04 20:53:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_775(8127720) done Nov 04 20:53:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1995(8151433) WEXITSTATUS 0 Nov 04 20:53:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1995(8151433) done Nov 04 20:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3456(8157325) WEXITSTATUS 0 Nov 04 20:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3456(8157325) done Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES cpu grp_used_tres_run_secs underflow, tried to remove 212 seconds when only 196 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES mem grp_used_tres_run_secs underflow, tried to remove 3256320 seconds when only 2732032 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES node grp_used_tres_run_secs underflow, tried to remove 53 seconds when only 37 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES billing grp_used_tres_run_secs underflow, tried to remove 795 seconds when only 555 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 53 seconds when only 37 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES cpu grp_used_tres_run_secs underflow, tried to remove 212 seconds when only 196 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES mem grp_used_tres_run_secs underflow, tried to remove 3256320 seconds when only 2732032 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES node grp_used_tres_run_secs underflow, tried to remove 53 seconds when only 37 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES billing grp_used_tres_run_secs underflow, tried to remove 795 seconds when only 555 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 53 seconds when only 37 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES cpu grp_used_tres_run_secs underflow, tried to remove 212 seconds when only 196 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES mem grp_used_tres_run_secs underflow, tried to remove 3256320 seconds when only 2732032 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES node grp_used_tres_run_secs underflow, tried to remove 53 seconds when only 37 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES billing grp_used_tres_run_secs underflow, tried to remove 795 seconds when only 555 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 5695 TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 53 seconds when only 37 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES cpu grp_used_tres_run_secs underflow, tried to remove 212 seconds when only 196 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES mem grp_used_tres_run_secs underflow, tried to remove 3256320 seconds when only 2732032 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES node grp_used_tres_run_secs underflow, tried to remove 53 seconds when only 37 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES billing grp_used_tres_run_secs underflow, tried to remove 795 seconds when only 555 remained. Nov 04 20:53:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8138481: assoc 2787 TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 53 seconds when only 37 remained. Nov 04 20:53:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4438(8155679) WEXITSTATUS 1 Nov 04 20:53:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4438(8155679) done Nov 04 20:53:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8138481 Nov 04 20:53:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4771(8157453) WEXITSTATUS 1 Nov 04 20:53:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4771(8157453) done Nov 04 20:53:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1182(8133749) WEXITSTATUS 0 Nov 04 20:53:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1182(8133749) done Nov 04 20:53:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8154451 uid 21920 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3470(8157737) in gpu on r808u08n01 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3471(8157738) in gpu on r808u14n01 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3472(8157739) in gpu on r808u17n01 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1737(8157740) in day on r806u14n03 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1738(8157741) in day on r807u11n03 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1739(8157742) in day on r904u07n03 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1740(8157743) in day on r904u04n01 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1741(8157744) in day on r914u05n02 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1742(8157745) in day on r918u09n03 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1743(8157746) in day on r909u07n04 Nov 04 20:53:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1744(8157747) in day on r909u29n01 Nov 04 20:53:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u03n03 Nov 04 20:53:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156044 in scavenge on r816u09n02 Nov 04 20:53:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1073(7573242) WEXITSTATUS 0 Nov 04 20:53:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1073(7573242) done Nov 04 20:53:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_734(7572903) WEXITSTATUS 0 Nov 04 20:53:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_734(7572903) done Nov 04 20:53:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144123 in scavenge on r915u38n[03-04] Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4802(8157748) in scavenge on r802u06n04 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4803(8157749) in scavenge on r915u38n02 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4804(8157750) in scavenge on r915u38n02 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4805(8157751) in scavenge on r915u38n02 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4806(8157752) in scavenge on r903u40n01 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4807(8157753) in scavenge on r907u24n01 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4808(8157754) in scavenge on r909u13n03 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4809(8157755) in scavenge on r918u09n03 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4363(8155089) WEXITSTATUS 1 Nov 04 20:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4363(8155089) done Nov 04 20:54:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1777(8150988) WEXITSTATUS 0 Nov 04 20:54:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1777(8150988) done Nov 04 20:54:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES cpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES mem grp_used_tres_run_secs underflow, tried to remove 491520 seconds when only 475136 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES cpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES billing grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES cpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES mem grp_used_tres_run_secs underflow, tried to remove 491520 seconds when only 475136 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES billing grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES mem grp_used_tres_run_secs underflow, tried to remove 491520 seconds when only 475136 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES billing grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES cpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES mem grp_used_tres_run_secs underflow, tried to remove 491520 seconds when only 475136 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES billing grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 40 remained. Nov 04 20:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3472(8157739) WEXITSTATUS 0 Nov 04 20:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3472(8157739) done Nov 04 20:54:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8157756 NodeList=(null) usec=36117 Nov 04 20:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157756 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 20:54:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907375 WEXITSTATUS 0 Nov 04 20:54:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907375 done Nov 04 20:54:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1877(8151203) WEXITSTATUS 0 Nov 04 20:54:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1877(8151203) done Nov 04 20:54:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1570(8149615) in scavenge on r915u38n02 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4435(8155676) WEXITSTATUS 1 Nov 04 20:54:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4435(8155676) done Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8147104 Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES cpu grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES mem grp_used_tres_run_secs underflow, tried to remove 131072 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES node grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES billing grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES cpu grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES mem grp_used_tres_run_secs underflow, tried to remove 131072 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES node grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES billing grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES cpu grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES mem grp_used_tres_run_secs underflow, tried to remove 131072 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES node grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 4721 TRES billing grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES cpu grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES mem grp_used_tres_run_secs underflow, tried to remove 131072 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES node grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8147104: assoc 72 TRES billing grp_used_tres_run_secs underflow, tried to remove 16 seconds when only 0 remained. Nov 04 20:54:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_854(8127968) WEXITSTATUS 0 Nov 04 20:54:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_854(8127968) done Nov 04 20:54:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_966(8128311) WEXITSTATUS 0 Nov 04 20:54:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_966(8128311) done Nov 04 20:54:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1867(8151193) WEXITSTATUS 0 Nov 04 20:54:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1867(8151193) done Nov 04 20:55:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4769(8157451) WEXITSTATUS 1 Nov 04 20:55:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4769(8157451) done Nov 04 20:55:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4679(8156341) WEXITSTATUS 1 Nov 04 20:55:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4679(8156341) done Nov 04 20:55:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 20:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8149444 WEXITSTATUS 0 Nov 04 20:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8149444 done Nov 04 20:55:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4502(8155885) WEXITSTATUS 1 Nov 04 20:55:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4502(8155885) done Nov 04 20:55:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_790(8127754) WEXITSTATUS 0 Nov 04 20:55:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_790(8127754) done Nov 04 20:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4389(8155319) WEXITSTATUS 1 Nov 04 20:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4389(8155319) done Nov 04 20:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4417(8155556) WEXITSTATUS 1 Nov 04 20:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4417(8155556) done Nov 04 20:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1996(8151434) WEXITSTATUS 0 Nov 04 20:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1996(8151434) done Nov 04 20:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4066(8147919) WEXITSTATUS 1 Nov 04 20:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4066(8147919) done Nov 04 20:56:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8156122 Nov 04 20:56:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3927(8145651) WEXITSTATUS 1 Nov 04 20:56:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3927(8145651) done Nov 04 20:56:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150058 WEXITSTATUS 0 Nov 04 20:56:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150058 done Nov 04 20:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_969(8128318) WEXITSTATUS 0 Nov 04 20:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_969(8128318) done Nov 04 20:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_36(8135735) WEXITSTATUS 0 Nov 04 20:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_36(8135735) done Nov 04 20:56:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157757 InitPrio=22846 usec=3687 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:56:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1873(8151199) WEXITSTATUS 0 Nov 04 20:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1873(8151199) done Nov 04 20:56:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3430(8156840) WEXITSTATUS 0 Nov 04 20:56:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3430(8156840) done Nov 04 20:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1160(8132967) WEXITSTATUS 0 Nov 04 20:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1160(8132967) done Nov 04 20:56:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3466(8157655) WEXITSTATUS 0 Nov 04 20:56:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3466(8157655) done Nov 04 20:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3453(8157322) WEXITSTATUS 0 Nov 04 20:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3453(8157322) done Nov 04 20:56:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3622(8136987) WEXITSTATUS 1 Nov 04 20:56:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3622(8136987) done Nov 04 20:57:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157757 in gpu on r808u17n01 Nov 04 20:57:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907376 in gpu on r805u38n01 Nov 04 20:57:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3473(8157758) in gpu on r903u34n01 Nov 04 20:57:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3474(8157759) in gpu on r808u11n01 Nov 04 20:57:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3475(8157760) in gpu on r4513u29n01 Nov 04 20:57:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3476(8157761) in gpu on r907u32n01 Nov 04 20:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1745(8157762) in day on r807u05n01 Nov 04 20:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1746(8157763) in day on r914u05n01 Nov 04 20:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1747(8157764) in day on r909u01n04 Nov 04 20:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1748(8157765) in day on r909u29n03 Nov 04 20:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1749(8157766) in day on r816u15n01 Nov 04 20:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156045 in scavenge on r815u23n07 Nov 04 20:57:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4810(8157767) in scavenge on r802u06n01 Nov 04 20:57:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4811(8157768) in scavenge on r903u04n03 Nov 04 20:57:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4812(8157769) in scavenge on r903u24n01 Nov 04 20:57:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4813(8157770) in scavenge on r906u41n01 Nov 04 20:57:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4814(8157771) in scavenge on r909u10n01 Nov 04 20:57:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4815(8157772) in scavenge on r918u05n03 Nov 04 20:57:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4816(8157773) in scavenge on r919u32n01 Nov 04 20:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1862(8151188) WEXITSTATUS 0 Nov 04 20:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1862(8151188) done Nov 04 20:57:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3426(8156710) WEXITSTATUS 0 Nov 04 20:57:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3426(8156710) done Nov 04 20:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135788 WEXITSTATUS 0 Nov 04 20:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135788 done Nov 04 20:57:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3474(8157759) WEXITSTATUS 0 Nov 04 20:57:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3474(8157759) done Nov 04 20:57:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2135(8151811) WEXITSTATUS 0 Nov 04 20:57:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2135(8151811) done Nov 04 20:57:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3476(8157761) WEXITSTATUS 0 Nov 04 20:57:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3476(8157761) done Nov 04 20:58:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1129(8131630) WEXITSTATUS 0 Nov 04 20:58:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1129(8131630) done Nov 04 20:58:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4315(8154198) WEXITSTATUS 1 Nov 04 20:58:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4315(8154198) done Nov 04 20:58:14 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_526(7571491) Nov 04 20:58:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_925(8128158) WEXITSTATUS 0 Nov 04 20:58:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_925(8128158) done Nov 04 20:58:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133241 WEXITSTATUS 0 Nov 04 20:58:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133241 done Nov 04 20:58:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157774 InitPrio=32537 usec=3769 Nov 04 20:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1170(8133633) WEXITSTATUS 0 Nov 04 20:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1170(8133633) done Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 20:58:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 20:58:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3456(8135312) WEXITSTATUS 1 Nov 04 20:58:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3456(8135312) done Nov 04 20:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4790(8157665) WEXITSTATUS 1 Nov 04 20:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4790(8157665) done Nov 04 20:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1999(8151437) WEXITSTATUS 0 Nov 04 20:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1999(8151437) done Nov 04 20:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2146(8151822) WEXITSTATUS 0 Nov 04 20:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2146(8151822) done Nov 04 20:59:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157341 WEXITSTATUS 0 Nov 04 20:59:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157341 done Nov 04 20:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_908(8128087) WEXITSTATUS 0 Nov 04 20:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_908(8128087) done Nov 04 20:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2234(8152026) WEXITSTATUS 0 Nov 04 20:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2234(8152026) done Nov 04 20:59:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156046 NodeList=r816u29n11 #CPUs=12 Partition=scavenge Nov 04 20:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4313(8154196) WEXITSTATUS 1 Nov 04 20:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4313(8154196) done Nov 04 20:59:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_47(8135746) WEXITSTATUS 0 Nov 04 20:59:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_47(8135746) done Nov 04 20:59:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1174(8133637) WEXITSTATUS 0 Nov 04 20:59:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1174(8133637) done Nov 04 20:59:37 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8157775 NodeList=(null) usec=21257 Nov 04 20:59:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157775 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 04 20:59:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1159(8132966) WEXITSTATUS 0 Nov 04 20:59:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1159(8132966) done Nov 04 21:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 04 21:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 04 21:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 04 21:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 04 21:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 04 21:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133245 WEXITSTATUS 0 Nov 04 21:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133245 done Nov 04 21:00:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r806u06n02 Nov 04 21:00:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r807u03n03 Nov 04 21:00:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157774 in day on r805u23n04 Nov 04 21:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3477(8157776) in gpu on r808u11n01 Nov 04 21:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3478(8157777) in gpu on r4513u23n01 Nov 04 21:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3479(8157778) in gpu on r907u32n01 Nov 04 21:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1750(8157779) in day on r914u05n03 Nov 04 21:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1751(8157780) in day on r918u01n02 Nov 04 21:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1752(8157781) in day on r909u07n02 Nov 04 21:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1753(8157782) in day on r909u13n02 Nov 04 21:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 21:00:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4782(8157657) WEXITSTATUS 1 Nov 04 21:00:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4782(8157657) done Nov 04 21:00:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_371(7572539) WEXITSTATUS 0 Nov 04 21:00:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_371(7572539) done Nov 04 21:00:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1240(8135133) WEXITSTATUS 0 Nov 04 21:00:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1240(8135133) done Nov 04 21:00:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1194(8134102) WEXITSTATUS 0 Nov 04 21:00:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1194(8134102) done Nov 04 21:00:14 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_0(8144687) Nov 04 21:00:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4817(8157783) in scavenge on r802u30n02 Nov 04 21:00:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4818(8157784) in scavenge on r903u04n01 Nov 04 21:00:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4819(8157785) in scavenge on r907u28n01 Nov 04 21:00:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4820(8157786) in scavenge on r918u01n02 Nov 04 21:00:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4821(8157787) in scavenge on r919u09n01 Nov 04 21:00:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_14(8157788) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:00:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135791 WEXITSTATUS 0 Nov 04 21:00:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135791 done Nov 04 21:00:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1863(8151189) WEXITSTATUS 0 Nov 04 21:00:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1863(8151189) done Nov 04 21:00:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 04 21:00:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 04 21:00:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157756 WEXITSTATUS 0 Nov 04 21:00:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157756 done Nov 04 21:00:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4541(8155946) WEXITSTATUS 1 Nov 04 21:00:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4541(8155946) done Nov 04 21:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1197(8134210) WEXITSTATUS 0 Nov 04 21:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1197(8134210) done Nov 04 21:00:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1625(8124203) Nov 04 21:00:45 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2518(8127608) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:00:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_2(8144787) Nov 04 21:00:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_3(8144788) Nov 04 21:00:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2518(8127608) Nov 04 21:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4631(8156223) WEXITSTATUS 1 Nov 04 21:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4631(8156223) done Nov 04 21:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1779(8150990) WEXITSTATUS 0 Nov 04 21:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1779(8150990) done Nov 04 21:00:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4579(8156092) WEXITSTATUS 1 Nov 04 21:00:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4579(8156092) done Nov 04 21:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4791(8157687) WEXITSTATUS 1 Nov 04 21:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4791(8157687) done Nov 04 21:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4606(8156182) WEXITSTATUS 1 Nov 04 21:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4606(8156182) done Nov 04 21:01:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157789 InitPrio=31249 usec=6331 Nov 04 21:01:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4805(8157751) WEXITSTATUS 1 Nov 04 21:01:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4805(8157751) done Nov 04 21:01:16 r912u23n01 slurmctld[163760]: slurmctld: build_job_queue has run for 2000000 usec, exiting with 200 of 15022 jobs tested, 25 job-partition pairs added Nov 04 21:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4526(8155927) WEXITSTATUS 1 Nov 04 21:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4526(8155927) done Nov 04 21:01:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_6(8144811) Nov 04 21:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1509(8150354) WEXITSTATUS 0 Nov 04 21:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1509(8150354) done Nov 04 21:01:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:01:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_15(8157790) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:01:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1166(8133521) WEXITSTATUS 0 Nov 04 21:01:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1166(8133521) done Nov 04 21:01:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1635(8150725) WEXITSTATUS 0 Nov 04 21:01:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1635(8150725) done Nov 04 21:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1806(8151017) WEXITSTATUS 0 Nov 04 21:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1806(8151017) done Nov 04 21:02:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157789 NodeList=r903u37n01 #CPUs=1 Partition=gpu_devel Nov 04 21:02:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2144(8151820) WEXITSTATUS 0 Nov 04 21:02:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2144(8151820) done Nov 04 21:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1506(8150351) WEXITSTATUS 0 Nov 04 21:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1506(8150351) done Nov 04 21:02:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_7(8144812) Nov 04 21:02:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_16(8157791) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:02:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133243 WEXITSTATUS 0 Nov 04 21:02:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133243 done Nov 04 21:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4533(8155938) WEXITSTATUS 1 Nov 04 21:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4533(8155938) done Nov 04 21:02:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157792 InitPrio=39797 usec=3688 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:02:36 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:02:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4721(8156732) WEXITSTATUS 1 Nov 04 21:02:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4721(8156732) done Nov 04 21:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157793 InitPrio=57376 usec=4233 Nov 04 21:02:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157793 NodeList=r801u30n01 #CPUs=1 Partition=pi_lora Nov 04 21:02:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4610(8156189) WEXITSTATUS 1 Nov 04 21:02:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4610(8156189) done Nov 04 21:02:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4419(8155558) WEXITSTATUS 1 Nov 04 21:02:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4419(8155558) done Nov 04 21:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157792 in gpu_devel on r903u37n01 Nov 04 21:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8149759_17(8157794) in day on r806u06n02 Nov 04 21:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8149759_18(8157795) in day on r909u04n02 Nov 04 21:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1754(8157796) in day on r805u02n02 Nov 04 21:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1755(8157797) in day on r914u05n04 Nov 04 21:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1756(8157798) in day on r919u05n04 Nov 04 21:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1757(8157799) in day on r815u35n02 Nov 04 21:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2133(8151809) WEXITSTATUS 0 Nov 04 21:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2133(8151809) done Nov 04 21:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8157800 NodeList=(null) usec=67653 Nov 04 21:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157800 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 21:03:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1564(8150543) WEXITSTATUS 0 Nov 04 21:03:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1564(8150543) done Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_8(8144816) Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4822(8157801) in scavenge on r907u09n03 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4823(8157802) in scavenge on r801u06n02 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4824(8157803) in scavenge on r912u05n04 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4825(8157804) in scavenge on r802u23n02 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4826(8157805) in scavenge on r802u32n04 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4827(8157806) in scavenge on r915u38n02 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4828(8157807) in scavenge on r917u10n01 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4829(8157808) in scavenge on r906u28n01 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4830(8157809) in scavenge on r805u02n04 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4831(8157810) in scavenge on r909u04n02 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4832(8157811) in scavenge on r913u05n03 Nov 04 21:03:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4833(8157812) in scavenge on r919u32n01 Nov 04 21:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4298(8154064) WEXITSTATUS 1 Nov 04 21:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4298(8154064) done Nov 04 21:03:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_19(8157813) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:03:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1878(8124878) WEXITSTATUS 1 Nov 04 21:03:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1878(8124878) done Nov 04 21:03:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1187(8133861) WEXITSTATUS 0 Nov 04 21:03:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1187(8133861) done Nov 04 21:03:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8156199 uid 26528 Nov 04 21:03:46 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8139113 Nov 04 21:03:46 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_10(8144929) Nov 04 21:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157814 InitPrio=31346 usec=3499 Nov 04 21:03:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_20(8157815) NodeList=r908u33n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157816 InitPrio=33841 usec=3666 Nov 04 21:03:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157816 NodeList=r815u29n08 #CPUs=1 Partition=pi_mingarelli Nov 04 21:03:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_0(8157548) WEXITSTATUS 0 Nov 04 21:03:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_0(8157548) done Nov 04 21:04:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1208(8134439) WEXITSTATUS 0 Nov 04 21:04:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1208(8134439) done Nov 04 21:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3467(8157672) WEXITSTATUS 0 Nov 04 21:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3467(8157672) done Nov 04 21:04:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8139226 Nov 04 21:04:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_11(8144954) Nov 04 21:04:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_12(8144955) Nov 04 21:04:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_13(8144956) Nov 04 21:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_21(8157817) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_22(8157818) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_23(8157819) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_24(8157820) NodeList=r908u22n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_938(8128198) WEXITSTATUS 0 Nov 04 21:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_938(8128198) done Nov 04 21:04:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157821 InitPrio=27499 usec=5085 Nov 04 21:04:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157822 InitPrio=50486 usec=6524 Nov 04 21:04:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3479(8157778) WEXITSTATUS 0 Nov 04 21:04:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3479(8157778) done Nov 04 21:04:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1900(8151226) WEXITSTATUS 0 Nov 04 21:04:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1900(8151226) done Nov 04 21:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144116 WEXITSTATUS 0 Nov 04 21:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144116 done Nov 04 21:05:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1247(8135149) WEXITSTATUS 0 Nov 04 21:05:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1247(8135149) done Nov 04 21:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157822 NodeList=r815u29n06 #CPUs=10 Partition=day Nov 04 21:05:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_15(8144960) Nov 04 21:05:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_25(8157823) NodeList=r908u39n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:05:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_411(8113301) WEXITSTATUS 1 Nov 04 21:05:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_411(8113301) done Nov 04 21:05:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4625(8156209) WEXITSTATUS 1 Nov 04 21:05:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4625(8156209) done Nov 04 21:05:46 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_824(7571790) Nov 04 21:05:46 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1639(8124235) Nov 04 21:05:46 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_16(8144963) Nov 04 21:05:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_26(8157824) NodeList=r914u13n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:05:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157821 in day on r909u01n01 Nov 04 21:05:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3480(8157825) in gpu on r903u34n01 Nov 04 21:05:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3481(8157826) in gpu on r907u32n01 Nov 04 21:05:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1758(8157827) in day on r909u01n03 Nov 04 21:05:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1759(8157828) in day on r918u32n02 Nov 04 21:05:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1760(8157829) in day on r918u36n04 Nov 04 21:05:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1761(8157830) in day on r816u03n01 Nov 04 21:05:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r806u06n02 Nov 04 21:05:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156047 in scavenge on r816u15n11 Nov 04 21:05:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4346(8154956) WEXITSTATUS 1 Nov 04 21:05:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4346(8154956) done Nov 04 21:05:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144119 in scavenge on r915u38n01,r916u01n02 Nov 04 21:06:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4834(8157831) in scavenge on r802u30n03 Nov 04 21:06:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4835(8157832) in scavenge on r916u04n01 Nov 04 21:06:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4836(8157833) in scavenge on r805u14n04 Nov 04 21:06:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4837(8157834) in scavenge on r805u14n04 Nov 04 21:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4509(8155898) WEXITSTATUS 1 Nov 04 21:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4509(8155898) done Nov 04 21:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3461(8157445) WEXITSTATUS 0 Nov 04 21:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3461(8157445) done Nov 04 21:06:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8139227 Nov 04 21:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1002(8128400) WEXITSTATUS 0 Nov 04 21:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1002(8128400) done Nov 04 21:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4741(8156981) WEXITSTATUS 1 Nov 04 21:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4741(8156981) done Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1597(8150576) WEXITSTATUS 0 Nov 04 21:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1597(8150576) done Nov 04 21:06:46 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_0(8145078) Nov 04 21:06:46 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_17(8145066) Nov 04 21:06:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_27(8157835) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:06:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157836 InitPrio=28352 usec=6118 Nov 04 21:06:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8157221 uid 23968 Nov 04 21:07:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8151450 uid 25715 Nov 04 21:07:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_18(8145083) Nov 04 21:07:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_19(8145084) Nov 04 21:07:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:07:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_28(8157837) NodeList=r908u37n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:07:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_29(8157838) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:07:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1158(8132853) WEXITSTATUS 0 Nov 04 21:07:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1158(8132853) done Nov 04 21:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1803(8151014) WEXITSTATUS 0 Nov 04 21:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1803(8151014) done Nov 04 21:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3481(8157826) WEXITSTATUS 0 Nov 04 21:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3481(8157826) done Nov 04 21:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4634(8156226) WEXITSTATUS 1 Nov 04 21:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4634(8156226) done Nov 04 21:07:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157839 InitPrio=57382 usec=3966 Nov 04 21:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_871(7571837) WEXITSTATUS 0 Nov 04 21:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_871(7571837) done Nov 04 21:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144119 WEXITSTATUS 0 Nov 04 21:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144119 done Nov 04 21:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144123 WEXITSTATUS 0 Nov 04 21:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144123 done Nov 04 21:07:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1647(8124250) Nov 04 21:07:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8124248 Nov 04 21:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4816(8157773) WEXITSTATUS 1 Nov 04 21:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4816(8157773) done Nov 04 21:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4829(8157808) WEXITSTATUS 1 Nov 04 21:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4829(8157808) done Nov 04 21:08:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1248(8135150) WEXITSTATUS 0 Nov 04 21:08:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1248(8135150) done Nov 04 21:08:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3475(8157760) WEXITSTATUS 0 Nov 04 21:08:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3475(8157760) done Nov 04 21:08:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_421(7571386) Nov 04 21:08:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_568(7571533) Nov 04 21:08:16 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1570(8149615) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:08:16 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4803(8157749) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:08:16 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4804(8157750) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:08:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4803(8157749) Nov 04 21:08:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4804(8157750) Nov 04 21:08:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1570(8149615) Nov 04 21:08:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157839 NodeList=r915u38n[01-04],r917u04n01 #CPUs=216 Partition=pi_lora Nov 04 21:08:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 21:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157340 WEXITSTATUS 0 Nov 04 21:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157340 done Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:08:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4498(8155881) WEXITSTATUS 1 Nov 04 21:08:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4498(8155881) done Nov 04 21:08:46 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_21(8145087) Nov 04 21:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_30(8157840) NodeList=r914u13n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:08:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157836 in gpu on r808u08n01 Nov 04 21:09:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3482(8157841) in gpu on r4513u29n01 Nov 04 21:09:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3483(8157842) in gpu on r907u32n01 Nov 04 21:09:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3484(8157843) in gpu on r808u17n01 Nov 04 21:09:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1762(8157844) in day on r919u01n04 Nov 04 21:09:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1763(8157845) in day on r919u36n03 Nov 04 21:09:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156048 in scavenge on r816u23n12 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4838(8157846) in scavenge on r903u07n01 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4839(8157847) in scavenge on r802u11n02 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4840(8157848) in scavenge on r802u32n01 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4841(8157849) in scavenge on r916u04n03 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4842(8157850) in scavenge on r917u13n04 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4843(8157851) in scavenge on r906u28n01 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4844(8157852) in scavenge on r805u23n03 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4845(8157853) in scavenge on r805u23n03 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4846(8157854) in scavenge on r805u23n03 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4847(8157855) in scavenge on r807u08n03 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4848(8157856) in scavenge on r909u10n01 Nov 04 21:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4849(8157857) in scavenge on r919u32n01 Nov 04 21:09:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8149759_31(8157858) in scavenge on r801u32n03 Nov 04 21:09:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_906(8128080) WEXITSTATUS 0 Nov 04 21:09:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_906(8128080) done Nov 04 21:09:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4589(8156110) WEXITSTATUS 1 Nov 04 21:09:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4589(8156110) done Nov 04 21:09:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2119(8151795) WEXITSTATUS 0 Nov 04 21:09:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2119(8151795) done Nov 04 21:09:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1569(8149614) in scavenge on r912u09n01 Nov 04 21:09:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1041(8128645) in scavenge on r915u09n01 Nov 04 21:09:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8157814 uid 21862 Nov 04 21:09:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8157793 uid 22479 Nov 04 21:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1819(8151145) WEXITSTATUS 0 Nov 04 21:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1819(8151145) done Nov 04 21:09:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_22(8145191) Nov 04 21:09:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_32(8157859) NodeList=r908u35n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:09:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1226(8135028) WEXITSTATUS 0 Nov 04 21:09:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1226(8135028) done Nov 04 21:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1897(8151223) WEXITSTATUS 0 Nov 04 21:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1897(8151223) done Nov 04 21:10:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2341(8152258) WEXITSTATUS 0 Nov 04 21:10:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2341(8152258) done Nov 04 21:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1186(8133860) WEXITSTATUS 0 Nov 04 21:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1186(8133860) done Nov 04 21:10:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4304(8154070) WEXITSTATUS 1 Nov 04 21:10:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4304(8154070) done Nov 04 21:10:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155871 WEXITSTATUS 0 Nov 04 21:10:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155871 done Nov 04 21:10:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8140414 Nov 04 21:10:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_23(8145199) Nov 04 21:10:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_24(8145200) Nov 04 21:10:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_25(8145201) Nov 04 21:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_33(8157860) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_34(8157861) NodeList=r915u29n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_35(8157862) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:10:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4628(8156212) WEXITSTATUS 1 Nov 04 21:10:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4628(8156212) done Nov 04 21:10:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3440(8156970) WEXITSTATUS 0 Nov 04 21:10:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3440(8156970) done Nov 04 21:10:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157863 InitPrio=31346 usec=3652 Nov 04 21:10:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3482(8157841) WEXITSTATUS 0 Nov 04 21:10:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3482(8157841) done Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:10:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:10:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157864 InitPrio=36191 usec=8162 Nov 04 21:10:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157864 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 21:10:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1951(8151389) WEXITSTATUS 0 Nov 04 21:10:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1951(8151389) done Nov 04 21:10:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4686(8156348) WEXITSTATUS 1 Nov 04 21:10:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4686(8156348) done Nov 04 21:10:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1691(8150781) WEXITSTATUS 0 Nov 04 21:10:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1691(8150781) done Nov 04 21:11:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1880(8151206) WEXITSTATUS 0 Nov 04 21:11:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1880(8151206) done Nov 04 21:11:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4694(8156466) WEXITSTATUS 1 Nov 04 21:11:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4694(8156466) done Nov 04 21:11:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_26(8145202) Nov 04 21:11:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_36(8157865) NodeList=r908u37n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:11:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157866 InitPrio=20041 usec=3915 Nov 04 21:11:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157342 WEXITSTATUS 0 Nov 04 21:11:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157342 done Nov 04 21:11:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3841(8144566) WEXITSTATUS 1 Nov 04 21:11:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3841(8144566) done Nov 04 21:11:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1662(8124282) Nov 04 21:11:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1665(8124285) Nov 04 21:11:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_27(8145203) Nov 04 21:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4843(8157851) WEXITSTATUS 1 Nov 04 21:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4843(8157851) done Nov 04 21:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1214(8134550) WEXITSTATUS 0 Nov 04 21:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1214(8134550) done Nov 04 21:11:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_37(8157867) NodeList=r908u37n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:11:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1167(8133522) WEXITSTATUS 0 Nov 04 21:11:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1167(8133522) done Nov 04 21:12:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157866 in day on r815u03n[02-03,05-06],r815u09n[02,05,07],r815u15n[02-05],r815u23n05 Nov 04 21:12:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3485(8157868) in gpu on r4513u20n01 Nov 04 21:12:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3486(8157869) in gpu on r4513u29n01 Nov 04 21:12:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1764(8157870) in day on r805u04n01 Nov 04 21:12:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1765(8157871) in day on r806u09n04 Nov 04 21:12:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1766(8157872) in day on r904u07n04 Nov 04 21:12:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1767(8157873) in day on r909u13n03 Nov 04 21:12:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1768(8157874) in day on r909u13n04 Nov 04 21:12:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r806u06n02 Nov 04 21:12:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156049 in scavenge on r816u03n04 Nov 04 21:12:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_935(8128194) WEXITSTATUS 0 Nov 04 21:12:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_935(8128194) done Nov 04 21:12:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3480(8157825) WEXITSTATUS 0 Nov 04 21:12:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3480(8157825) done Nov 04 21:12:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4850(8157875) in scavenge on r903u01n01 Nov 04 21:12:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4851(8157876) in scavenge on r916u01n01 Nov 04 21:12:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4852(8157877) in scavenge on r906u28n01 Nov 04 21:12:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4853(8157878) in scavenge on r805u27n04 Nov 04 21:12:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4854(8157879) in scavenge on r909u13n04 Nov 04 21:12:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4855(8157880) in scavenge on r805u36n01 Nov 04 21:12:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8149759_38(8157881) in scavenge on r917u07n01 Nov 04 21:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1851(8151177) WEXITSTATUS 0 Nov 04 21:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1851(8151177) done Nov 04 21:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1188(8133862) WEXITSTATUS 0 Nov 04 21:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1188(8133862) done Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:12:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:12:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8157863 uid 21862 Nov 04 21:12:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_3(8145302) Nov 04 21:12:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_28(8145303) Nov 04 21:12:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_39(8157882) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1834(8151160) WEXITSTATUS 0 Nov 04 21:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1834(8151160) done Nov 04 21:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4439(8155680) WEXITSTATUS 1 Nov 04 21:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4439(8155680) done Nov 04 21:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1985(8151423) WEXITSTATUS 0 Nov 04 21:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1985(8151423) done Nov 04 21:13:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4591(8156112) WEXITSTATUS 1 Nov 04 21:13:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4591(8156112) done Nov 04 21:13:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1615(8150594) WEXITSTATUS 0 Nov 04 21:13:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1615(8150594) done Nov 04 21:13:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1853(8151179) WEXITSTATUS 0 Nov 04 21:13:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1853(8151179) done Nov 04 21:13:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_883(8128020) WEXITSTATUS 0 Nov 04 21:13:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_883(8128020) done Nov 04 21:13:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_869(7571835) Nov 04 21:13:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8124291 Nov 04 21:13:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_29(8145320) Nov 04 21:13:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_30(8145321) Nov 04 21:13:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_31(8145322) Nov 04 21:13:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157883 InitPrio=15924 usec=10128 Nov 04 21:13:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:13:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157884 InitPrio=15924 usec=4871 Nov 04 21:13:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157885 InitPrio=15924 usec=3821 Nov 04 21:13:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_40(8157886) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:13:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_41(8157887) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:13:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_42(8157888) NodeList=r915u29n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:13:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157889 InitPrio=15924 usec=3892 Nov 04 21:13:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157890 InitPrio=15924 usec=3823 Nov 04 21:13:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157891 InitPrio=15924 usec=4922 Nov 04 21:13:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157892 InitPrio=15924 usec=5270 Nov 04 21:13:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157893 InitPrio=15924 usec=5878 Nov 04 21:13:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157894 InitPrio=15924 usec=3958 Nov 04 21:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2027(8151587) WEXITSTATUS 0 Nov 04 21:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2027(8151587) done Nov 04 21:13:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157895 InitPrio=15924 usec=5060 Nov 04 21:13:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157896 InitPrio=15924 usec=4911 Nov 04 21:13:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157897 InitPrio=15924 usec=4263 Nov 04 21:13:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157898 InitPrio=15924 usec=4025 Nov 04 21:13:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157899 InitPrio=15924 usec=3999 Nov 04 21:13:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157900 InitPrio=15924 usec=3918 Nov 04 21:13:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157901 InitPrio=15924 usec=3888 Nov 04 21:13:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157902 InitPrio=15924 usec=3828 Nov 04 21:13:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157903 InitPrio=31346 usec=3526 Nov 04 21:13:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157904 InitPrio=15924 usec=3803 Nov 04 21:13:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157905 InitPrio=15924 usec=3860 Nov 04 21:13:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157906 InitPrio=15924 usec=3998 Nov 04 21:13:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157907 InitPrio=15924 usec=3049 Nov 04 21:13:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157908 InitPrio=15924 usec=3720 Nov 04 21:13:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157909 InitPrio=15924 usec=3738 Nov 04 21:13:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157910 InitPrio=15924 usec=3717 Nov 04 21:13:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157911 InitPrio=15924 usec=3718 Nov 04 21:13:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157912 InitPrio=15924 usec=3735 Nov 04 21:14:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1542(8150521) WEXITSTATUS 0 Nov 04 21:14:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1542(8150521) done Nov 04 21:14:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157913 InitPrio=15924 usec=5066 Nov 04 21:14:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157914 InitPrio=15924 usec=4682 Nov 04 21:14:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157915 InitPrio=15924 usec=4287 Nov 04 21:14:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157916 InitPrio=15924 usec=3390 Nov 04 21:14:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157917 InitPrio=15924 usec=4045 Nov 04 21:14:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157918 InitPrio=15924 usec=3743 Nov 04 21:14:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3464(8157653) WEXITSTATUS 0 Nov 04 21:14:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3464(8157653) done Nov 04 21:14:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157919 InitPrio=15924 usec=3758 Nov 04 21:14:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157920 InitPrio=15924 usec=3833 Nov 04 21:14:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_32(8145323) Nov 04 21:14:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157921 InitPrio=15924 usec=3906 Nov 04 21:14:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157922 InitPrio=15924 usec=5016 Nov 04 21:14:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_43(8157923) NodeList=r914u13n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:14:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157924 InitPrio=15924 usec=3955 Nov 04 21:14:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157925 InitPrio=15924 usec=3885 Nov 04 21:14:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 04 21:14:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 04 21:14:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 04 21:14:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1912(8151350) WEXITSTATUS 0 Nov 04 21:14:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1912(8151350) done Nov 04 21:14:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157926 InitPrio=15924 usec=4025 Nov 04 21:14:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157927 InitPrio=15924 usec=4007 Nov 04 21:14:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157928 InitPrio=15924 usec=4042 Nov 04 21:14:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157929 InitPrio=15924 usec=3843 Nov 04 21:14:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157930 InitPrio=15924 usec=3986 Nov 04 21:14:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157931 InitPrio=15924 usec=3802 Nov 04 21:14:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3464(8135361) WEXITSTATUS 1 Nov 04 21:14:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3464(8135361) done Nov 04 21:14:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157932 InitPrio=15924 usec=3787 Nov 04 21:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3469(8157674) WEXITSTATUS 0 Nov 04 21:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3469(8157674) done Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:14:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:14:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1175(8133638) WEXITSTATUS 0 Nov 04 21:14:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1175(8133638) done Nov 04 21:14:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4161(8150608) WEXITSTATUS 1 Nov 04 21:14:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4161(8150608) done Nov 04 21:15:04 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8149759_44(8157933) in day on r805u06n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3487(8157934) in gpu on r808u17n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3488(8157935) in gpu on r903u34n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3489(8157936) in gpu on r903u34n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157883 in day on r906u41n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157884 in day on r906u41n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157885 in day on r906u41n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157889 in day on r906u41n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157890 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157891 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157892 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157893 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157894 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157895 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157896 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157897 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157898 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157899 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157900 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157901 in day on r807u03n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157902 in day on r807u03n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157904 in day on r807u03n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157905 in day on r904u07n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157906 in day on r904u07n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157907 in day on r904u07n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157908 in day on r904u07n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157909 in day on r904u07n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157910 in day on r904u07n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157911 in day on r904u07n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157912 in day on r906u41n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157913 in day on r806u04n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157914 in day on r806u06n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157915 in day on r806u06n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157916 in day on r806u06n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157917 in day on r806u11n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157918 in day on r806u23n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157919 in day on r912u28n04 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157920 in day on r912u28n04 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157921 in day on r913u01n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157922 in day on r913u01n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157924 in day on r913u01n04 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157925 in day on r913u09n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157926 in day on r913u13n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157927 in day on r913u24n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157928 in day on r913u24n04 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157929 in day on r913u24n04 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157930 in day on r913u28n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157931 in day on r913u28n04 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157932 in day on r805u02n04 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1769(8157937) in day on r909u07n03 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1770(8157938) in day on r909u10n02 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1771(8157939) in day on r918u05n01 Nov 04 21:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1772(8157940) in day on r918u36n02 Nov 04 21:15:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156050 in scavenge on r816u15n09 Nov 04 21:15:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156051 in scavenge on r815u29n04 Nov 04 21:15:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156052 in scavenge on r815u29n04 Nov 04 21:15:06 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:112 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 21:15:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1911(8151349) WEXITSTATUS 0 Nov 04 21:15:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1911(8151349) done Nov 04 21:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1994(8151432) WEXITSTATUS 0 Nov 04 21:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1994(8151432) done Nov 04 21:15:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3393(8155663) WEXITSTATUS 0 Nov 04 21:15:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3393(8155663) done Nov 04 21:15:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4856(8157941) in scavenge on r802u06n01 Nov 04 21:15:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4857(8157942) in scavenge on r917u01n02 Nov 04 21:15:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4858(8157943) in scavenge on r909u29n02 Nov 04 21:15:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4859(8157944) in scavenge on r919u32n01 Nov 04 21:15:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8141696 Nov 04 21:15:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_33(8145324) Nov 04 21:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1969(8151407) WEXITSTATUS 0 Nov 04 21:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1969(8151407) done Nov 04 21:15:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:900 Nov 04 21:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1954(8151392) WEXITSTATUS 0 Nov 04 21:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1954(8151392) done Nov 04 21:15:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_45(8157945) NodeList=r908u39n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4371(8155097) WEXITSTATUS 1 Nov 04 21:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4371(8155097) done Nov 04 21:15:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4038(8147091) WEXITSTATUS 1 Nov 04 21:15:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4038(8147091) done Nov 04 21:15:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157946 InitPrio=77209 usec=3845 Nov 04 21:15:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157946 NodeList=r802u14n03 #CPUs=1 Partition=pi_manohar Nov 04 21:15:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_698(7571663) Nov 04 21:15:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_35(8145423) Nov 04 21:15:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_36(8145424) Nov 04 21:15:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_37(8145425) Nov 04 21:15:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4637(8156229) WEXITSTATUS 1 Nov 04 21:15:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4637(8156229) done Nov 04 21:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2036(8151596) WEXITSTATUS 0 Nov 04 21:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2036(8151596) done Nov 04 21:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1627(8150717) WEXITSTATUS 0 Nov 04 21:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1627(8150717) done Nov 04 21:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1201(8134325) WEXITSTATUS 0 Nov 04 21:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1201(8134325) done Nov 04 21:16:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156053 NodeList=r816u03n10 #CPUs=12 Partition=scavenge Nov 04 21:16:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_38(8145477) Nov 04 21:16:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_39(8145478) Nov 04 21:16:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_40(8145499) Nov 04 21:16:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_46(8157947) NodeList=r908u37n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:16:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_47(8157948) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:16:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_48(8157949) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135790 WEXITSTATUS 0 Nov 04 21:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135790 done Nov 04 21:16:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8155686 uid 26133 Nov 04 21:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3473(8157758) WEXITSTATUS 0 Nov 04 21:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3473(8157758) done Nov 04 21:16:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4859(8157944) WEXITSTATUS 1 Nov 04 21:16:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4859(8157944) done Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:16:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3470(8157737) WEXITSTATUS 0 Nov 04 21:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3470(8157737) done Nov 04 21:16:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8141697 Nov 04 21:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1808(8151019) WEXITSTATUS 0 Nov 04 21:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1808(8151019) done Nov 04 21:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3489(8157936) WEXITSTATUS 0 Nov 04 21:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3489(8157936) done Nov 04 21:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1064(8129776) WEXITSTATUS 0 Nov 04 21:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1064(8129776) done Nov 04 21:17:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_41(8145544) Nov 04 21:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4457(8155770) WEXITSTATUS 1 Nov 04 21:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4457(8155770) done Nov 04 21:17:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8149759_49(8149759) NodeList=r908u35n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 21:17:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157950 InitPrio=37817 usec=3920 Nov 04 21:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3401(8156174) WEXITSTATUS 0 Nov 04 21:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3401(8156174) done Nov 04 21:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3484(8157843) WEXITSTATUS 0 Nov 04 21:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3484(8157843) done Nov 04 21:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2069(8151629) WEXITSTATUS 0 Nov 04 21:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2069(8151629) done Nov 04 21:17:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_42(8145545) Nov 04 21:17:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8156737 uid 22216 Nov 04 21:17:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157950 in day on r805u06n03 Nov 04 21:17:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3490(8157951) in gpu on r808u08n01 Nov 04 21:17:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3491(8157952) in gpu on r808u14n01 Nov 04 21:17:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3492(8157953) in gpu on r903u34n01 Nov 04 21:17:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3493(8157954) in gpu on r903u34n01 Nov 04 21:17:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3494(8157955) in gpu on r903u40n01 Nov 04 21:17:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3495(8157956) in gpu on r808u17n01 Nov 04 21:17:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1773(8157957) in day on r805u09n03 Nov 04 21:17:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1774(8157958) in day on r815u35n05 Nov 04 21:17:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r806u04n03 Nov 04 21:17:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156054 in scavenge on r815u23n05 Nov 04 21:17:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156055 in scavenge on r815u29n02 Nov 04 21:17:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156056 in scavenge on r815u29n02 Nov 04 21:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4834(8157831) WEXITSTATUS 1 Nov 04 21:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4834(8157831) done Nov 04 21:18:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8157339 uid 20536 Nov 04 21:18:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4860(8157959) in scavenge on r908u35n02 Nov 04 21:18:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4861(8157960) in scavenge on r908u35n02 Nov 04 21:18:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4862(8157961) in scavenge on r801u06n01 Nov 04 21:18:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4863(8157962) in scavenge on r919u32n01 Nov 04 21:18:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4864(8157963) in scavenge on r805u06n03 Nov 04 21:18:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4865(8157964) in scavenge on r805u09n03 Nov 04 21:18:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4866(8157965) in scavenge on r806u31n01 Nov 04 21:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4536(8155941) WEXITSTATUS 1 Nov 04 21:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4536(8155941) done Nov 04 21:18:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157966 InitPrio=59582 usec=3721 Nov 04 21:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157966 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 21:18:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1454(8137179) in scavenge on r906u13n01 Nov 04 21:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3493(8157954) WEXITSTATUS 0 Nov 04 21:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3493(8157954) done Nov 04 21:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1533(8150512) WEXITSTATUS 0 Nov 04 21:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1533(8150512) done Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:18:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:18:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1970(8151408) WEXITSTATUS 0 Nov 04 21:18:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1970(8151408) done Nov 04 21:18:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_44(8145646) Nov 04 21:18:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_45(8145648) Nov 04 21:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3488(8157935) WEXITSTATUS 0 Nov 04 21:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3488(8157935) done Nov 04 21:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_900(8128068) WEXITSTATUS 0 Nov 04 21:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_900(8128068) done Nov 04 21:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4450(8155763) WEXITSTATUS 1 Nov 04 21:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4450(8155763) done Nov 04 21:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_915(8128122) WEXITSTATUS 0 Nov 04 21:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_915(8128122) done Nov 04 21:19:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156057 NodeList=r816u35n10 #CPUs=12 Partition=scavenge Nov 04 21:19:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_47(8145662) Nov 04 21:19:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4733(8156862) WEXITSTATUS 1 Nov 04 21:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4733(8156862) done Nov 04 21:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1909(8151235) WEXITSTATUS 0 Nov 04 21:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1909(8151235) done Nov 04 21:19:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157967 InitPrio=37817 usec=4232 Nov 04 21:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4553(8155962) WEXITSTATUS 1 Nov 04 21:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4553(8155962) done Nov 04 21:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157967 NodeList=r918u32n02 #CPUs=4 Partition=day Nov 04 21:20:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8157836 uid 25715 Nov 04 21:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4480(8155853) WEXITSTATUS 1 Nov 04 21:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4480(8155853) done Nov 04 21:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4851(8157876) WEXITSTATUS 1 Nov 04 21:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4851(8157876) done Nov 04 21:20:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144686_49(8144686) Nov 04 21:20:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_890(8128037) WEXITSTATUS 0 Nov 04 21:20:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_890(8128037) done Nov 04 21:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133242 WEXITSTATUS 0 Nov 04 21:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133242 done Nov 04 21:20:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_9(8135708) WEXITSTATUS 0 Nov 04 21:20:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_9(8135708) done Nov 04 21:20:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131621 WEXITSTATUS 0 Nov 04 21:20:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131621 done Nov 04 21:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4778(8157460) WEXITSTATUS 1 Nov 04 21:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4778(8157460) done Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:20:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3483(8157842) WEXITSTATUS 0 Nov 04 21:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3483(8157842) done Nov 04 21:20:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3496(8157968) in gpu on r903u34n01 Nov 04 21:20:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3497(8157969) in gpu on r903u34n01 Nov 04 21:20:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1775(8157970) in day on r816u03n01 Nov 04 21:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156058 in scavenge on r815u09n03 Nov 04 21:20:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_4(8145669) Nov 04 21:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4720(8156731) WEXITSTATUS 1 Nov 04 21:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4720(8156731) done Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4867(8157971) in scavenge on r908u39n03 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4868(8157972) in scavenge on r908u39n03 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4869(8157973) in scavenge on r908u33n02 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4870(8157974) in scavenge on r908u33n02 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4871(8157975) in scavenge on r916u01n01 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4872(8157976) in scavenge on r802u11n01 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4873(8157977) in scavenge on r802u30n03 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4874(8157978) in scavenge on r805u09n03 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4875(8157979) in scavenge on r918u32n02 Nov 04 21:20:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4876(8157980) in scavenge on r918u32n02 Nov 04 21:20:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1162(7572129) in scavenge on r912u13n01 Nov 04 21:21:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4377(8155204) WEXITSTATUS 1 Nov 04 21:21:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4377(8155204) done Nov 04 21:21:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_944(7571910) Nov 04 21:21:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1700(8124425) Nov 04 21:21:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_870(8127984) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:21:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_871(8127985) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:21:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1428(8135961) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:21:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1428(8135961) Nov 04 21:21:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_870(8127984) Nov 04 21:21:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_871(8127985) Nov 04 21:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1958(8151396) WEXITSTATUS 0 Nov 04 21:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1958(8151396) done Nov 04 21:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4751(8157101) WEXITSTATUS 1 Nov 04 21:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4751(8157101) done Nov 04 21:21:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_789(8127753) WEXITSTATUS 0 Nov 04 21:21:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_789(8127753) done Nov 04 21:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1548(8150527) WEXITSTATUS 0 Nov 04 21:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1548(8150527) done Nov 04 21:21:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1702(8124430) Nov 04 21:21:48 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_5(8145768) Nov 04 21:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1948(8151386) WEXITSTATUS 0 Nov 04 21:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1948(8151386) done Nov 04 21:21:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3496(8157968) WEXITSTATUS 0 Nov 04 21:21:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3496(8157968) done Nov 04 21:21:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1978(8151416) WEXITSTATUS 0 Nov 04 21:21:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1978(8151416) done Nov 04 21:21:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_845(8127942) WEXITSTATUS 0 Nov 04 21:21:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_845(8127942) done Nov 04 21:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8149446 WEXITSTATUS 0 Nov 04 21:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8149446 done Nov 04 21:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1924(8151362) WEXITSTATUS 0 Nov 04 21:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1924(8151362) done Nov 04 21:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157903 NodeList=r816u35n07,r907u32n01 #CPUs=8 Partition=gpu Nov 04 21:22:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_6(8145781) Nov 04 21:22:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_7(8145782) Nov 04 21:22:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_8(8145783) Nov 04 21:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1899(8151225) WEXITSTATUS 0 Nov 04 21:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1899(8151225) done Nov 04 21:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4251(8153127) WEXITSTATUS 1 Nov 04 21:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4251(8153127) done Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150169 WEXITSTATUS 0 Nov 04 21:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150169 done Nov 04 21:22:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2023(8151583) WEXITSTATUS 0 Nov 04 21:22:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2023(8151583) done Nov 04 21:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_921(8128144) WEXITSTATUS 0 Nov 04 21:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_921(8128144) done Nov 04 21:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_950(8128277) WEXITSTATUS 0 Nov 04 21:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_950(8128277) done Nov 04 21:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4786(8157661) WEXITSTATUS 1 Nov 04 21:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4786(8157661) done Nov 04 21:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2224(8152016) WEXITSTATUS 0 Nov 04 21:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2224(8152016) done Nov 04 21:23:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4646(8156250) WEXITSTATUS 1 Nov 04 21:23:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4646(8156250) done Nov 04 21:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4848(8157856) WEXITSTATUS 1 Nov 04 21:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4848(8157856) done Nov 04 21:23:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4630(8156222) WEXITSTATUS 1 Nov 04 21:23:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4630(8156222) done Nov 04 21:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4425(8155666) WEXITSTATUS 1 Nov 04 21:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4425(8155666) done Nov 04 21:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2045(8151605) WEXITSTATUS 0 Nov 04 21:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2045(8151605) done Nov 04 21:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133244 WEXITSTATUS 0 Nov 04 21:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133244 done Nov 04 21:23:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3498(8157981) in gpu on r808u08n01 Nov 04 21:23:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3499(8157982) in gpu on r808u08n01 Nov 04 21:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1776(8157983) in day on r909u29n01 Nov 04 21:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1777(8157984) in day on r909u29n02 Nov 04 21:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1778(8157985) in day on r918u32n01 Nov 04 21:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1779(8157986) in day on r919u01n02 Nov 04 21:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r806u04n03 Nov 04 21:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156059 in scavenge on r815u35n03 Nov 04 21:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156060 in scavenge on r815u35n03 Nov 04 21:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157774 WEXITSTATUS 0 Nov 04 21:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157774 done Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8157987 InitPrio=37174 usec=6540 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4877(8157988) in scavenge on r908u39n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4878(8157989) in scavenge on r907u05n03 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4879(8157990) in scavenge on r908u35n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4880(8157991) in scavenge on r908u35n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4881(8157992) in scavenge on r908u33n03 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4882(8157993) in scavenge on r908u33n03 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4883(8157994) in scavenge on r908u33n04 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4884(8157995) in scavenge on r908u33n04 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4885(8157996) in scavenge on r801u23n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4886(8157997) in scavenge on r802u04n03 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4887(8157998) in scavenge on r911u05n02 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4888(8157999) in scavenge on r808u08n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4889(8158000) in scavenge on r907u24n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4890(8158001) in scavenge on r805u09n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4891(8158002) in scavenge on r919u01n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4892(8158003) in scavenge on r909u10n01 Nov 04 21:23:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4893(8158004) in scavenge on r909u13n04 Nov 04 21:23:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1166(7572133) in scavenge on r908u39n01 Nov 04 21:23:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1026(7571992) Nov 04 21:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4492(8155865) WEXITSTATUS 1 Nov 04 21:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4492(8155865) done Nov 04 21:23:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158005 InitPrio=37173 usec=4017 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:23:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:23:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2186(8151978) WEXITSTATUS 0 Nov 04 21:23:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2186(8151978) done Nov 04 21:23:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2046(8151606) WEXITSTATUS 0 Nov 04 21:23:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2046(8151606) done Nov 04 21:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157903 WEXITSTATUS 0 Nov 04 21:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157903 done Nov 04 21:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2125(8151801) WEXITSTATUS 0 Nov 04 21:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2125(8151801) done Nov 04 21:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3494(8157955) WEXITSTATUS 0 Nov 04 21:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3494(8157955) done Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_1(8158006) NodeList=r805u23n04 #CPUs=8 Partition=day Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_2(8158007) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_441(7572609) WEXITSTATUS 0 Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_441(7572609) done Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158008 InitPrio=37173 usec=5126 Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1075(7573244) WEXITSTATUS 0 Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1075(7573244) done Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1913(8151351) WEXITSTATUS 0 Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1913(8151351) done Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_576(7572745) WEXITSTATUS 0 Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_576(7572745) done Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_18(7572185) WEXITSTATUS 0 Nov 04 21:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_18(7572185) done Nov 04 21:24:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4420(8155559) WEXITSTATUS 1 Nov 04 21:24:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4420(8155559) done Nov 04 21:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_2(8158007) WEXITSTATUS 1 Nov 04 21:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_2(8158007) done Nov 04 21:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_1(8158006) WEXITSTATUS 1 Nov 04 21:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_1(8158006) done Nov 04 21:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2005(8151565) WEXITSTATUS 0 Nov 04 21:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2005(8151565) done Nov 04 21:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4818(8157784) WEXITSTATUS 1 Nov 04 21:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4818(8157784) done Nov 04 21:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2086(8151762) WEXITSTATUS 0 Nov 04 21:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2086(8151762) done Nov 04 21:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4841(8157849) WEXITSTATUS 1 Nov 04 21:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4841(8157849) done Nov 04 21:24:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8141905 Nov 04 21:24:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8141908 Nov 04 21:24:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8142324 Nov 04 21:24:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_9(8145881) Nov 04 21:24:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4362(8155088) WEXITSTATUS 1 Nov 04 21:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4362(8155088) done Nov 04 21:24:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158009 InitPrio=28352 usec=3895 Nov 04 21:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3485(8157868) WEXITSTATUS 0 Nov 04 21:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3485(8157868) done Nov 04 21:25:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157987 NodeList=r807u39n01 #CPUs=20 Partition=bigmem Nov 04 21:25:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_3(8158010) NodeList=r805u23n04 #CPUs=8 Partition=day Nov 04 21:25:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_4(8158011) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:25:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_5(8158012) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:25:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156061 NodeList=r815u15n11 #CPUs=12 Partition=scavenge Nov 04 21:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_600(7571565) WEXITSTATUS 0 Nov 04 21:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_600(7571565) done Nov 04 21:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1535(8150514) WEXITSTATUS 0 Nov 04 21:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1535(8150514) done Nov 04 21:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_4(8158011) WEXITSTATUS 1 Nov 04 21:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_4(8158011) done Nov 04 21:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_5(8158012) WEXITSTATUS 1 Nov 04 21:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_5(8158012) done Nov 04 21:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_3(8158010) WEXITSTATUS 1 Nov 04 21:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_3(8158010) done Nov 04 21:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157987 WEXITSTATUS 1 Nov 04 21:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157987 done Nov 04 21:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_901(8128071) WEXITSTATUS 0 Nov 04 21:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_901(8128071) done Nov 04 21:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1196(8134209) WEXITSTATUS 0 Nov 04 21:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1196(8134209) done Nov 04 21:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4885(8157996) WEXITSTATUS 1 Nov 04 21:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4885(8157996) done Nov 04 21:25:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_856(8127970) WEXITSTATUS 0 Nov 04 21:25:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_856(8127970) done Nov 04 21:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4489(8155862) WEXITSTATUS 1 Nov 04 21:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4489(8155862) done Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1669(8150759) WEXITSTATUS 0 Nov 04 21:25:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1669(8150759) done Nov 04 21:25:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4485(8155858) WEXITSTATUS 1 Nov 04 21:25:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4485(8155858) done Nov 04 21:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1778(8124619) WEXITSTATUS 1 Nov 04 21:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1778(8124619) done Nov 04 21:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4659(8156321) WEXITSTATUS 1 Nov 04 21:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4659(8156321) done Nov 04 21:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4640(8156244) WEXITSTATUS 1 Nov 04 21:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4640(8156244) done Nov 04 21:26:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_6(8158013) NodeList=r805u23n04 #CPUs=8 Partition=day Nov 04 21:26:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_7(8158014) NodeList=r906u41n03 #CPUs=8 Partition=day Nov 04 21:26:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_8(8158015) NodeList=r918u36n02 #CPUs=8 Partition=day Nov 04 21:26:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_9(8158016) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:26:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_10(8158017) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:26:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158008 NodeList=r912u09n01 #CPUs=8 Partition=bigmem Nov 04 21:26:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156062 NodeList=r816u35n01 #CPUs=12 Partition=scavenge Nov 04 21:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_9(8158016) WEXITSTATUS 1 Nov 04 21:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_9(8158016) done Nov 04 21:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_10(8158017) WEXITSTATUS 1 Nov 04 21:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_10(8158017) done Nov 04 21:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_6(8158013) WEXITSTATUS 1 Nov 04 21:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_6(8158013) done Nov 04 21:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3892(8145071) WEXITSTATUS 1 Nov 04 21:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3892(8145071) done Nov 04 21:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158008 WEXITSTATUS 1 Nov 04 21:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158008 done Nov 04 21:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_7(8158014) WEXITSTATUS 1 Nov 04 21:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_7(8158014) done Nov 04 21:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_8(8158015) WEXITSTATUS 1 Nov 04 21:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_8(8158015) done Nov 04 21:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4516(8155905) WEXITSTATUS 1 Nov 04 21:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4516(8155905) done Nov 04 21:26:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1069(7572035) Nov 04 21:26:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158018 InitPrio=34324 usec=3238 Nov 04 21:26:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158018 NodeList=r801u06n01 #CPUs=4 Partition=education Nov 04 21:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2075(8151635) WEXITSTATUS 0 Nov 04 21:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2075(8151635) done Nov 04 21:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1945(8151383) WEXITSTATUS 0 Nov 04 21:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1945(8151383) done Nov 04 21:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1242(8135135) WEXITSTATUS 0 Nov 04 21:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1242(8135135) done Nov 04 21:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158005_11(8158019) in day on r805u23n04 Nov 04 21:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158005_12(8158020) in day on r906u41n03 Nov 04 21:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158005_13(8158021) in day on r909u29n01 Nov 04 21:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158005_14(8158022) in day on r918u36n02 Nov 04 21:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158005_15(8158023) in day on r815u03n07 Nov 04 21:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158005_16(8158024) in day on r815u03n07 Nov 04 21:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158009 in gpu on r805u38n01 Nov 04 21:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3500(8158025) in gpu on r4513u20n01 Nov 04 21:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3501(8158026) in gpu on r903u34n01 Nov 04 21:26:26 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:900 Nov 04 21:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_15(8158023) WEXITSTATUS 1 Nov 04 21:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_15(8158023) done Nov 04 21:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_16(8158024) WEXITSTATUS 1 Nov 04 21:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_16(8158024) done Nov 04 21:26:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156063 in scavenge on r815u03n06 Nov 04 21:26:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156064 in scavenge on r815u23n06 Nov 04 21:26:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156065 in scavenge on r816u15n01 Nov 04 21:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_14(8158022) WEXITSTATUS 1 Nov 04 21:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_14(8158022) done Nov 04 21:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_12(8158020) WEXITSTATUS 1 Nov 04 21:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_12(8158020) done Nov 04 21:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_11(8158019) WEXITSTATUS 1 Nov 04 21:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_11(8158019) done Nov 04 21:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_13(8158021) WEXITSTATUS 1 Nov 04 21:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_13(8158021) done Nov 04 21:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4866(8157965) WEXITSTATUS 1 Nov 04 21:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4866(8157965) done Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4894(8158027) in scavenge on r801u23n01 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4895(8158028) in scavenge on r802u02n03 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4896(8158029) in scavenge on r903u04n01 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4897(8158030) in scavenge on r916u04n04 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4898(8158031) in scavenge on r916u04n03 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4899(8158032) in scavenge on r808u08n01 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4900(8158033) in scavenge on r912u28n01 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4901(8158034) in scavenge on r912u32n03 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4902(8158035) in scavenge on r912u32n03 Nov 04 21:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4903(8158036) in scavenge on r913u13n01 Nov 04 21:26:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_33(7572200) in scavenge on r912u09n01 Nov 04 21:26:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1780(8158037) in scavenge on r816u35n07 Nov 04 21:26:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1781(8158038) in scavenge on r816u35n07 Nov 04 21:26:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1782(8158039) in scavenge on r816u35n07 Nov 04 21:26:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2041(8151601) WEXITSTATUS 0 Nov 04 21:26:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2041(8151601) done Nov 04 21:26:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146596_4(8146596) WEXITSTATUS 0 Nov 04 21:26:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146596_4(8146596) done Nov 04 21:26:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2185(8151977) WEXITSTATUS 0 Nov 04 21:26:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2185(8151977) done Nov 04 21:27:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158009 uid 25715 Nov 04 21:27:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_17(8158040) NodeList=r805u23n04 #CPUs=8 Partition=day Nov 04 21:27:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_18(8158041) NodeList=r906u41n03 #CPUs=8 Partition=day Nov 04 21:27:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_19(8158042) NodeList=r909u29n01 #CPUs=8 Partition=day Nov 04 21:27:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158005_20(8158005) NodeList=r918u36n02 #CPUs=8 Partition=day Nov 04 21:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_18(8158041) WEXITSTATUS 1 Nov 04 21:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_18(8158041) done Nov 04 21:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_19(8158042) WEXITSTATUS 1 Nov 04 21:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_19(8158042) done Nov 04 21:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_20(8158005) WEXITSTATUS 1 Nov 04 21:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_20(8158005) done Nov 04 21:27:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_17(8158040) WEXITSTATUS 1 Nov 04 21:27:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158005_17(8158040) done Nov 04 21:27:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158043 InitPrio=34324 usec=3461 Nov 04 21:27:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158043 NodeList=r801u06n02 #CPUs=4 Partition=education Nov 04 21:27:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1157(8123276) WEXITSTATUS 1 Nov 04 21:27:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1157(8123276) done Nov 04 21:27:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158044 InitPrio=28352 usec=3792 Nov 04 21:27:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8142531 Nov 04 21:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25734: QOSMaxSubmitJobPerUserLimit Nov 04 21:27:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 21:27:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_10(8145988) Nov 04 21:27:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_11(8145989) Nov 04 21:27:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_12(8145990) Nov 04 21:27:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_13(8145991) Nov 04 21:27:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_14(8145992) Nov 04 21:27:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_15(8145993) Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:27:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4412(8155450) WEXITSTATUS 1 Nov 04 21:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4412(8155450) done Nov 04 21:28:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156066 NodeList=r816u23n08 #CPUs=12 Partition=scavenge Nov 04 21:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3971(8146120) WEXITSTATUS 1 Nov 04 21:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3971(8146120) done Nov 04 21:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4735(8156864) WEXITSTATUS 1 Nov 04 21:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4735(8156864) done Nov 04 21:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_927(8128169) WEXITSTATUS 0 Nov 04 21:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_927(8128169) done Nov 04 21:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_1(8157549) WEXITSTATUS 0 Nov 04 21:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_1(8157549) done Nov 04 21:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1649(8150739) WEXITSTATUS 0 Nov 04 21:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1649(8150739) done Nov 04 21:28:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1025(7571991) Nov 04 21:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4535(8155940) WEXITSTATUS 1 Nov 04 21:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4535(8155940) done Nov 04 21:29:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158045 InitPrio=13783 usec=3627 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158044 in gpu on r805u38n01 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158045 in gpu_devel on r914u30n01 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1783(8158046) in day on r805u23n04 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1784(8158047) in day on r906u41n03 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1785(8158048) in day on r909u29n01 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1786(8158049) in day on r918u36n02 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1787(8158050) in day on r919u09n04 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1788(8158051) in day on r815u23n01 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1789(8158052) in day on r815u09n04 Nov 04 21:29:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1790(8158053) in day on r815u09n08 Nov 04 21:29:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n03 Nov 04 21:29:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156067 in scavenge on r815u23n01 Nov 04 21:29:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156068 in scavenge on r815u09n04 Nov 04 21:29:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156069 in scavenge on r815u09n04 Nov 04 21:29:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156070 in scavenge on r815u09n08 Nov 04 21:29:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4904(8158054) in scavenge on r911u05n01 Nov 04 21:29:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4905(8158055) in scavenge on r918u05n02 Nov 04 21:29:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4906(8158056) in scavenge on r918u05n02 Nov 04 21:29:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4907(8158057) in scavenge on r918u05n02 Nov 04 21:29:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4908(8158058) in scavenge on r806u31n01 Nov 04 21:29:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4909(8158059) in scavenge on r906u32n01 Nov 04 21:29:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4899(8158032) WEXITSTATUS 1 Nov 04 21:29:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4899(8158032) done Nov 04 21:29:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4380(8155310) WEXITSTATUS 1 Nov 04 21:29:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4380(8155310) done Nov 04 21:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3495(8157956) WEXITSTATUS 0 Nov 04 21:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3495(8157956) done Nov 04 21:29:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3487(8157934) WEXITSTATUS 0 Nov 04 21:29:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3487(8157934) done Nov 04 21:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3471(8157738) WEXITSTATUS 0 Nov 04 21:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3471(8157738) done Nov 04 21:29:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8142743 Nov 04 21:29:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8142949 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:29:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:29:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158060 InitPrio=37173 usec=5096 Nov 04 21:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4166(8150812) WEXITSTATUS 1 Nov 04 21:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4166(8150812) done Nov 04 21:30:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_1(8158061) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:30:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_2(8158062) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2002(8151562) WEXITSTATUS 0 Nov 04 21:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2002(8151562) done Nov 04 21:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_1(8158061) WEXITSTATUS 1 Nov 04 21:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_1(8158061) done Nov 04 21:30:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_2(8158062) WEXITSTATUS 1 Nov 04 21:30:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_2(8158062) done Nov 04 21:30:21 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_16(8146116) Nov 04 21:30:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4780(8157462) WEXITSTATUS 1 Nov 04 21:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4780(8157462) done Nov 04 21:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2201(8126074) WEXITSTATUS 1 Nov 04 21:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2201(8126074) done Nov 04 21:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4504(8155887) WEXITSTATUS 1 Nov 04 21:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4504(8155887) done Nov 04 21:30:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4777(8157459) WEXITSTATUS 1 Nov 04 21:30:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4777(8157459) done Nov 04 21:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4877(8157988) WEXITSTATUS 1 Nov 04 21:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4877(8157988) done Nov 04 21:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4546(8155955) WEXITSTATUS 1 Nov 04 21:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4546(8155955) done Nov 04 21:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3501(8158026) WEXITSTATUS 0 Nov 04 21:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3501(8158026) done Nov 04 21:31:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_3(8158063) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:31:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_4(8158064) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:31:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_5(8158065) NodeList=r815u09n01 #CPUs=8 Partition=day Nov 04 21:31:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_944(8128240) WEXITSTATUS 0 Nov 04 21:31:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_944(8128240) done Nov 04 21:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4584(8156097) WEXITSTATUS 1 Nov 04 21:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4584(8156097) done Nov 04 21:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_3(8158063) WEXITSTATUS 1 Nov 04 21:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_3(8158063) done Nov 04 21:31:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_4(8158064) WEXITSTATUS 1 Nov 04 21:31:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_4(8158064) done Nov 04 21:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_5(8158065) WEXITSTATUS 1 Nov 04 21:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_5(8158065) done Nov 04 21:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1818(8151144) WEXITSTATUS 0 Nov 04 21:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1818(8151144) done Nov 04 21:31:23 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_849(7571815) Nov 04 21:31:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4597(8156118) WEXITSTATUS 1 Nov 04 21:31:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4597(8156118) done Nov 04 21:31:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4292(8154058) WEXITSTATUS 1 Nov 04 21:31:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4292(8154058) done Nov 04 21:31:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4369(8155095) WEXITSTATUS 1 Nov 04 21:31:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4369(8155095) done Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146147 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:31:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1210(8134441) WEXITSTATUS 0 Nov 04 21:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1210(8134441) done Nov 04 21:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8157105 uid 22405 Nov 04 21:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158060_6(8158066) in day on r815u03n07 Nov 04 21:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158060_7(8158067) in day on r815u03n07 Nov 04 21:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158060_8(8158068) in day on r815u09n01 Nov 04 21:32:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r904u04n03 Nov 04 21:32:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r904u10n01 Nov 04 21:32:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3502(8158069) in gpu on r808u14n01 Nov 04 21:32:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3503(8158070) in gpu on r808u17n01 Nov 04 21:32:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3504(8158071) in gpu on r903u34n01 Nov 04 21:32:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3505(8158072) in gpu on r903u40n01 Nov 04 21:32:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3500(8158025) WEXITSTATUS 0 Nov 04 21:32:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3500(8158025) done Nov 04 21:32:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_6(8158066) WEXITSTATUS 1 Nov 04 21:32:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_6(8158066) done Nov 04 21:32:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_8(8158068) WEXITSTATUS 1 Nov 04 21:32:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_8(8158068) done Nov 04 21:32:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_7(8158067) WEXITSTATUS 1 Nov 04 21:32:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_7(8158067) done Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4910(8158073) in scavenge on r907u05n01 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4911(8158074) in scavenge on r908u39n01 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4912(8158075) in scavenge on r917u07n04 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4913(8158076) in scavenge on r802u06n02 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4914(8158077) in scavenge on r802u11n01 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4915(8158078) in scavenge on r808u08n01 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4916(8158079) in scavenge on r805u02n04 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4917(8158080) in scavenge on r919u09n04 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4918(8158081) in scavenge on r912u32n04 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4919(8158082) in scavenge on r912u36n03 Nov 04 21:32:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4920(8158083) in scavenge on r815u23n02 Nov 04 21:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1625(8150715) WEXITSTATUS 0 Nov 04 21:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1625(8150715) done Nov 04 21:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4632(8156224) WEXITSTATUS 1 Nov 04 21:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4632(8156224) done Nov 04 21:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4523(8155924) WEXITSTATUS 1 Nov 04 21:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4523(8155924) done Nov 04 21:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 04 21:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 04 21:32:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 04 21:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_934(8128190) WEXITSTATUS 0 Nov 04 21:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_934(8128190) done Nov 04 21:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1156(8132851) WEXITSTATUS 0 Nov 04 21:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1156(8132851) done Nov 04 21:32:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158084 InitPrio=59099 usec=3873 Nov 04 21:32:52 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3619(8136984) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:32:52 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4671(8156333) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:32:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3619(8136984) Nov 04 21:32:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4671(8156333) Nov 04 21:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3504(8158071) WEXITSTATUS 0 Nov 04 21:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3504(8158071) done Nov 04 21:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158084 NodeList=r904u07n02 #CPUs=1 Partition=day Nov 04 21:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_9(8158085) NodeList=r909u13n04 #CPUs=8 Partition=day Nov 04 21:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_10(8158086) NodeList=r919u36n01 #CPUs=8 Partition=day Nov 04 21:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_11(8158087) NodeList=r919u36n02 #CPUs=8 Partition=day Nov 04 21:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_12(8158088) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_13(8158089) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_14(8158090) NodeList=r815u09n01 #CPUs=8 Partition=day Nov 04 21:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3492(8157953) WEXITSTATUS 0 Nov 04 21:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3492(8157953) done Nov 04 21:33:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_12(8158088) WEXITSTATUS 1 Nov 04 21:33:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_12(8158088) done Nov 04 21:33:23 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_17(8146251) Nov 04 21:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_13(8158089) WEXITSTATUS 1 Nov 04 21:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_13(8158089) done Nov 04 21:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_14(8158090) WEXITSTATUS 1 Nov 04 21:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_14(8158090) done Nov 04 21:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_10(8158086) WEXITSTATUS 1 Nov 04 21:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_10(8158086) done Nov 04 21:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_9(8158085) WEXITSTATUS 1 Nov 04 21:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_9(8158085) done Nov 04 21:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_11(8158087) WEXITSTATUS 1 Nov 04 21:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_11(8158087) done Nov 04 21:33:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1667(8150757) WEXITSTATUS 0 Nov 04 21:33:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1667(8150757) done Nov 04 21:33:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4712(8156723) WEXITSTATUS 1 Nov 04 21:33:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4712(8156723) done Nov 04 21:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4604(8156180) WEXITSTATUS 1 Nov 04 21:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4604(8156180) done Nov 04 21:33:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4652(8156256) WEXITSTATUS 1 Nov 04 21:33:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4652(8156256) done Nov 04 21:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4718(8156729) WEXITSTATUS 1 Nov 04 21:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4718(8156729) done Nov 04 21:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2028(8151588) WEXITSTATUS 0 Nov 04 21:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2028(8151588) done Nov 04 21:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3517(8135648) WEXITSTATUS 1 Nov 04 21:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3517(8135648) done Nov 04 21:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4868(8157972) WEXITSTATUS 1 Nov 04 21:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4868(8157972) done Nov 04 21:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2116(8151792) WEXITSTATUS 0 Nov 04 21:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2116(8151792) done Nov 04 21:33:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158091 InitPrio=35354 usec=7215 Nov 04 21:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4673(8156335) WEXITSTATUS 1 Nov 04 21:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4673(8156335) done Nov 04 21:33:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_552(7571517) Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:33:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:34:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158092 InitPrio=34083 usec=5353 Nov 04 21:34:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1162(8133199) WEXITSTATUS 0 Nov 04 21:34:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1162(8133199) done Nov 04 21:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_15(8158093) NodeList=r909u13n04 #CPUs=8 Partition=day Nov 04 21:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_16(8158094) NodeList=r919u36n01 #CPUs=8 Partition=day Nov 04 21:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_17(8158095) NodeList=r919u36n02 #CPUs=8 Partition=day Nov 04 21:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_18(8158096) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_19(8158097) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158060_20(8158060) NodeList=r815u09n01 #CPUs=8 Partition=day Nov 04 21:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158091 NodeList=r904u10n01 #CPUs=1 Partition=day Nov 04 21:34:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:13 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:16 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:16 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:17 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1192(8133970) WEXITSTATUS 0 Nov 04 21:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1192(8133970) done Nov 04 21:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4521(8155922) WEXITSTATUS 1 Nov 04 21:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4521(8155922) done Nov 04 21:34:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_913(8128112) WEXITSTATUS 0 Nov 04 21:34:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_913(8128112) done Nov 04 21:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_20(8158060) WEXITSTATUS 1 Nov 04 21:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_20(8158060) done Nov 04 21:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_18(8158096) WEXITSTATUS 1 Nov 04 21:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_18(8158096) done Nov 04 21:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_19(8158097) WEXITSTATUS 1 Nov 04 21:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_19(8158097) done Nov 04 21:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_16(8158094) WEXITSTATUS 1 Nov 04 21:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_16(8158094) done Nov 04 21:34:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_15(8158093) WEXITSTATUS 1 Nov 04 21:34:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_15(8158093) done Nov 04 21:34:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_17(8158095) WEXITSTATUS 1 Nov 04 21:34:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158060_17(8158095) done Nov 04 21:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1832(8151158) WEXITSTATUS 0 Nov 04 21:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1832(8151158) done Nov 04 21:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4282(8154048) WEXITSTATUS 1 Nov 04 21:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4282(8154048) done Nov 04 21:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_6(8157465) WEXITSTATUS 0 Nov 04 21:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_6(8157465) done Nov 04 21:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3938(8145770) WEXITSTATUS 1 Nov 04 21:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3938(8145770) done Nov 04 21:34:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1936(8151374) WEXITSTATUS 0 Nov 04 21:34:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1936(8151374) done Nov 04 21:34:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158098 InitPrio=32537 usec=4348 Nov 04 21:35:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158092 in day on r807u11n02 Nov 04 21:35:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158098 in day on r909u01n04 Nov 04 21:35:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3506(8158099) in gpu on r4513u20n01 Nov 04 21:35:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3507(8158100) in gpu on r903u34n01 Nov 04 21:35:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3508(8158101) in gpu on r903u34n01 Nov 04 21:35:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1791(8158102) in day on r909u13n04 Nov 04 21:35:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1792(8158103) in day on r918u09n02 Nov 04 21:35:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1793(8158104) in day on r919u36n01 Nov 04 21:35:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1794(8158105) in day on r919u36n02 Nov 04 21:35:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1795(8158106) in day on r815u09n01 Nov 04 21:35:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n04 Nov 04 21:35:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156071 in scavenge on r815u03n09 Nov 04 21:35:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158107 InitPrio=37174 usec=4589 Nov 04 21:35:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_693(7572862) WEXITSTATUS 0 Nov 04 21:35:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_693(7572862) done Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4921(8158108) in scavenge on r801u30n01 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4922(8158109) in scavenge on r907u09n03 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4923(8158110) in scavenge on r908u39n03 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4924(8158111) in scavenge on r801u06n02 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4925(8158112) in scavenge on r802u02n03 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4926(8158113) in scavenge on r911u05n02 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4927(8158114) in scavenge on r916u04n02 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4928(8158115) in scavenge on r805u09n04 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4929(8158116) in scavenge on r909u13n04 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4930(8158117) in scavenge on r918u32n01 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4931(8158118) in scavenge on r919u32n01 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4932(8158119) in scavenge on r919u36n01 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4933(8158120) in scavenge on r919u36n01 Nov 04 21:35:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4934(8158121) in scavenge on r815u35n04 Nov 04 21:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158107 NodeList=r807u39n01 #CPUs=20 Partition=bigmem Nov 04 21:35:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155920 WEXITSTATUS 0 Nov 04 21:35:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155920 done Nov 04 21:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1610(8150589) WEXITSTATUS 0 Nov 04 21:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1610(8150589) done Nov 04 21:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1120(8131612) in scavenge on r801u06n01 Nov 04 21:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1570(8149615) in scavenge on r912u09n01 Nov 04 21:35:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158122 InitPrio=37173 usec=4315 Nov 04 21:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3506(8158099) WEXITSTATUS 0 Nov 04 21:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3506(8158099) done Nov 04 21:35:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2237(8152029) WEXITSTATUS 0 Nov 04 21:35:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2237(8152029) done Nov 04 21:35:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2001(8151561) WEXITSTATUS 0 Nov 04 21:35:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2001(8151561) done Nov 04 21:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2292(8152209) WEXITSTATUS 0 Nov 04 21:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2292(8152209) done Nov 04 21:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1956(8151394) WEXITSTATUS 0 Nov 04 21:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1956(8151394) done Nov 04 21:35:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158123 InitPrio=10370 usec=4051 Nov 04 21:35:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158124 InitPrio=37173 usec=4477 Nov 04 21:35:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158125 InitPrio=37173 usec=4237 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:35:55 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:35:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4464(8155781) WEXITSTATUS 1 Nov 04 21:35:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4464(8155781) done Nov 04 21:35:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1227(8135029) WEXITSTATUS 0 Nov 04 21:35:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1227(8135029) done Nov 04 21:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4601(8156177) WEXITSTATUS 1 Nov 04 21:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4601(8156177) done Nov 04 21:36:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1938(8151376) WEXITSTATUS 0 Nov 04 21:36:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1938(8151376) done Nov 04 21:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3507(8158100) WEXITSTATUS 0 Nov 04 21:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3507(8158100) done Nov 04 21:36:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_1(8158126) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:36:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_2(8158127) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:36:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_3(8158128) NodeList=r815u03n08 #CPUs=8 Partition=day Nov 04 21:36:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2371(8152415) WEXITSTATUS 0 Nov 04 21:36:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2371(8152415) done Nov 04 21:36:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1190(7572157) Nov 04 21:36:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_47(7572214) Nov 04 21:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_1(8158126) WEXITSTATUS 1 Nov 04 21:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_1(8158126) done Nov 04 21:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_2(8158127) WEXITSTATUS 1 Nov 04 21:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_2(8158127) done Nov 04 21:36:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_3(8158128) WEXITSTATUS 1 Nov 04 21:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_3(8158128) done Nov 04 21:36:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2092(8151768) WEXITSTATUS 0 Nov 04 21:36:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2092(8151768) done Nov 04 21:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4647(8156251) WEXITSTATUS 1 Nov 04 21:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4647(8156251) done Nov 04 21:36:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907585 WEXITSTATUS 0 Nov 04 21:36:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907585 done Nov 04 21:36:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2083(8151759) WEXITSTATUS 0 Nov 04 21:36:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2083(8151759) done Nov 04 21:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_918(8128125) WEXITSTATUS 0 Nov 04 21:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_918(8128125) done Nov 04 21:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4387(8155317) WEXITSTATUS 1 Nov 04 21:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4387(8155317) done Nov 04 21:36:52 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4675(8156337) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:36:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4675(8156337) Nov 04 21:37:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4140(8149756) WEXITSTATUS 1 Nov 04 21:37:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4140(8149756) done Nov 04 21:37:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_4(8158129) NodeList=r909u01n02 #CPUs=8 Partition=day Nov 04 21:37:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_5(8158130) NodeList=r919u05n01 #CPUs=8 Partition=day Nov 04 21:37:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_6(8158131) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:37:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_7(8158132) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:37:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_8(8158133) NodeList=r815u03n08 #CPUs=8 Partition=day Nov 04 21:37:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8156072 NodeList=r815u15n10 #CPUs=12 Partition=scavenge Nov 04 21:37:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_876(8128010) WEXITSTATUS 0 Nov 04 21:37:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_876(8128010) done Nov 04 21:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_6(8158131) WEXITSTATUS 1 Nov 04 21:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_6(8158131) done Nov 04 21:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_8(8158133) WEXITSTATUS 1 Nov 04 21:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_8(8158133) done Nov 04 21:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_7(8158132) WEXITSTATUS 1 Nov 04 21:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_7(8158132) done Nov 04 21:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_4(8158129) WEXITSTATUS 1 Nov 04 21:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_4(8158129) done Nov 04 21:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_5(8158130) WEXITSTATUS 1 Nov 04 21:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_5(8158130) done Nov 04 21:37:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3465(8157654) WEXITSTATUS 0 Nov 04 21:37:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3465(8157654) done Nov 04 21:37:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4351(8154961) WEXITSTATUS 1 Nov 04 21:37:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4351(8154961) done Nov 04 21:37:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1859(8151185) WEXITSTATUS 0 Nov 04 21:37:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1859(8151185) done Nov 04 21:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2065(8151625) WEXITSTATUS 0 Nov 04 21:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2065(8151625) done Nov 04 21:37:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158122_9(8158134) in day on r919u05n01 Nov 04 21:37:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158122_10(8158135) in day on r909u01n02 Nov 04 21:37:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158122_11(8158136) in day on r909u13n01 Nov 04 21:37:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158122_12(8158137) in day on r815u03n07 Nov 04 21:37:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158122_13(8158138) in day on r815u03n07 Nov 04 21:37:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158122_14(8158139) in day on r815u03n08 Nov 04 21:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907586 in gpu on r806u35n01 Nov 04 21:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3509(8158140) in gpu on r903u34n01 Nov 04 21:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3510(8158141) in gpu on r808u14n01 Nov 04 21:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3511(8158142) in gpu on r4513u20n01 Nov 04 21:37:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158143 InitPrio=13461 usec=2673 Nov 04 21:37:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158123 in day on r906u41n04 Nov 04 21:37:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4914(8158077) WEXITSTATUS 1 Nov 04 21:37:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4914(8158077) done Nov 04 21:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_12(8158137) WEXITSTATUS 1 Nov 04 21:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_12(8158137) done Nov 04 21:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_14(8158139) WEXITSTATUS 1 Nov 04 21:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_14(8158139) done Nov 04 21:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_13(8158138) WEXITSTATUS 1 Nov 04 21:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_13(8158138) done Nov 04 21:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_10(8158135) WEXITSTATUS 1 Nov 04 21:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_10(8158135) done Nov 04 21:38:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_9(8158134) WEXITSTATUS 1 Nov 04 21:38:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_9(8158134) done Nov 04 21:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4935(8158144) in scavenge on r903u01n01 Nov 04 21:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4936(8158145) in scavenge on r907u32n01 Nov 04 21:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4937(8158146) in scavenge on r918u36n02 Nov 04 21:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4938(8158147) in scavenge on r919u01n04 Nov 04 21:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4939(8158148) in scavenge on r904u13n01 Nov 04 21:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_11(8158136) WEXITSTATUS 1 Nov 04 21:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_11(8158136) done Nov 04 21:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_15(8158149) NodeList=r919u05n01 #CPUs=8 Partition=day Nov 04 21:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_16(8158150) NodeList=r909u01n02 #CPUs=8 Partition=day Nov 04 21:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_17(8158151) NodeList=r909u13n01 #CPUs=8 Partition=day Nov 04 21:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_18(8158152) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_19(8158153) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158122_20(8158122) NodeList=r815u03n08 #CPUs=8 Partition=day Nov 04 21:38:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1796(8158154) in scavenge on r904u13n01 Nov 04 21:38:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1797(8158155) in scavenge on r904u13n01 Nov 04 21:38:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1798(8158156) in scavenge on r904u13n01 Nov 04 21:38:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1799(8158157) in scavenge on r904u13n01 Nov 04 21:38:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1800(8158158) in scavenge on r904u13n01 Nov 04 21:38:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1801(8158159) in scavenge on r904u13n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:38:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_18(8158152) WEXITSTATUS 1 Nov 04 21:38:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_18(8158152) done Nov 04 21:38:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_20(8158122) WEXITSTATUS 1 Nov 04 21:38:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_20(8158122) done Nov 04 21:38:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_19(8158153) WEXITSTATUS 1 Nov 04 21:38:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_19(8158153) done Nov 04 21:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_17(8158151) WEXITSTATUS 1 Nov 04 21:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_17(8158151) done Nov 04 21:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_16(8158150) WEXITSTATUS 1 Nov 04 21:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_16(8158150) done Nov 04 21:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_15(8158149) WEXITSTATUS 1 Nov 04 21:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158122_15(8158149) done Nov 04 21:38:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1726(8150937) WEXITSTATUS 0 Nov 04 21:38:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1726(8150937) done Nov 04 21:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2196(8151988) WEXITSTATUS 0 Nov 04 21:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2196(8151988) done Nov 04 21:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3511(8158142) WEXITSTATUS 0 Nov 04 21:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3511(8158142) done Nov 04 21:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4112(8149227) WEXITSTATUS 1 Nov 04 21:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4112(8149227) done Nov 04 21:38:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3505(8158072) WEXITSTATUS 0 Nov 04 21:38:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3505(8158072) done Nov 04 21:38:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1096(7572062) Nov 04 21:38:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_75(7572242) Nov 04 21:38:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8128530 Nov 04 21:38:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158160 InitPrio=37173 usec=4008 Nov 04 21:39:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_1(8158161) NodeList=r919u05n01 #CPUs=8 Partition=day Nov 04 21:39:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_2(8158162) NodeList=r909u01n02 #CPUs=8 Partition=day Nov 04 21:39:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_3(8158163) NodeList=r909u13n01 #CPUs=8 Partition=day Nov 04 21:39:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_4(8158164) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:39:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_5(8158165) NodeList=r815u03n07 #CPUs=8 Partition=day Nov 04 21:39:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_6(8158166) NodeList=r815u03n08 #CPUs=8 Partition=day Nov 04 21:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4906(8158056) WEXITSTATUS 1 Nov 04 21:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4906(8158056) done Nov 04 21:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_933(8128182) WEXITSTATUS 0 Nov 04 21:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_933(8128182) done Nov 04 21:39:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158167 InitPrio=20041 usec=4555 Nov 04 21:39:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157866 WEXITSTATUS 0 Nov 04 21:39:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157866 done Nov 04 21:39:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1626(8150716) WEXITSTATUS 0 Nov 04 21:39:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1626(8150716) done Nov 04 21:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4890(8158001) WEXITSTATUS 1 Nov 04 21:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4890(8158001) done Nov 04 21:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4813(8157770) WEXITSTATUS 1 Nov 04 21:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4813(8157770) done Nov 04 21:40:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1215(8134551) WEXITSTATUS 0 Nov 04 21:40:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1215(8134551) done Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_7(8158168) NodeList=r919u01n03 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_8(8158169) NodeList=r919u05n02 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_9(8158170) NodeList=r815u23n05 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_10(8158171) NodeList=r815u03n02 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_11(8158172) NodeList=r815u03n03 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_12(8158173) NodeList=r815u03n05 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_13(8158174) NodeList=r815u03n06 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_14(8158175) NodeList=r815u09n02 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_15(8158176) NodeList=r815u09n05 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_16(8158177) NodeList=r815u09n07 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_17(8158178) NodeList=r815u15n02 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_18(8158179) NodeList=r815u15n03 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_19(8158180) NodeList=r815u15n04 #CPUs=8 Partition=day Nov 04 21:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158124_20(8158124) NodeList=r815u15n05 #CPUs=8 Partition=day Nov 04 21:40:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158143 uid 19028 Nov 04 21:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1061(8129773) WEXITSTATUS 0 Nov 04 21:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1061(8129773) done Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:40:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3509(8158140) WEXITSTATUS 0 Nov 04 21:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3509(8158140) done Nov 04 21:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4190(8151645) WEXITSTATUS 1 Nov 04 21:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4190(8151645) done Nov 04 21:40:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4644(8156248) WEXITSTATUS 1 Nov 04 21:40:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4644(8156248) done Nov 04 21:40:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8143682 Nov 04 21:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158181 InitPrio=35982 usec=8459 Nov 04 21:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157775 WTERMSIG 9 Nov 04 21:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157775 done Nov 04 21:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3512(8158182) in gpu on r4513u20n01 Nov 04 21:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3513(8158183) in gpu on r903u34n01 Nov 04 21:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3514(8158184) in gpu on r903u40n01 Nov 04 21:40:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157946 WEXITSTATUS 0 Nov 04 21:40:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157946 done Nov 04 21:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 21:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156073 in scavenge on r815u23n05 Nov 04 21:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156074 in scavenge on r815u03n05 Nov 04 21:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8156075 in scavenge on r815u09n02 Nov 04 21:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157466 in scavenge on r815u09n07 Nov 04 21:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157467 in scavenge on r815u15n03 Nov 04 21:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157468 in scavenge on r815u15n05 Nov 04 21:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1758(8150969) WEXITSTATUS 0 Nov 04 21:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1758(8150969) done Nov 04 21:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2230(8152022) WEXITSTATUS 0 Nov 04 21:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2230(8152022) done Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4940(8158185) in scavenge on r802u02n03 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4941(8158186) in scavenge on r802u04n03 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4942(8158187) in scavenge on r802u11n01 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4943(8158188) in scavenge on r815u29n09 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4944(8158189) in scavenge on r4513u23n01 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4945(8158190) in scavenge on r805u09n01 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4946(8158191) in scavenge on r906u41n01 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4947(8158192) in scavenge on r906u41n03 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4948(8158193) in scavenge on r906u41n03 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4949(8158194) in scavenge on r918u05n02 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4950(8158195) in scavenge on r815u15n06 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4951(8158196) in scavenge on r815u15n07 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4952(8158197) in scavenge on r815u15n08 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3514(8158184) WEXITSTATUS 0 Nov 04 21:41:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3514(8158184) done Nov 04 21:41:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 21:41:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_22(7572189) Nov 04 21:41:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_52(7572219) Nov 04 21:41:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_54(7572221) Nov 04 21:41:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1802(8158198) in scavenge on r911u05n02 Nov 04 21:41:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1803(8158199) in scavenge on r906u13n01 Nov 04 21:41:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3508(8158101) WEXITSTATUS 0 Nov 04 21:41:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3508(8158101) done Nov 04 21:41:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2231(8152023) WEXITSTATUS 0 Nov 04 21:41:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2231(8152023) done Nov 04 21:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3668(8138075) WEXITSTATUS 1 Nov 04 21:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3668(8138075) done Nov 04 21:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4479(8155852) WEXITSTATUS 1 Nov 04 21:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4479(8155852) done Nov 04 21:42:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157469 NodeList=r815u23n08 #CPUs=12 Partition=scavenge Nov 04 21:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4940(8158185) WEXITSTATUS 1 Nov 04 21:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4940(8158185) done Nov 04 21:42:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_18(8146587) Nov 04 21:42:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8155966 Nov 04 21:42:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:42:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1063(8129775) WEXITSTATUS 0 Nov 04 21:42:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1063(8129775) done Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:42:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:42:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4793(8157689) WEXITSTATUS 1 Nov 04 21:42:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4793(8157689) done Nov 04 21:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2120(8151796) WEXITSTATUS 0 Nov 04 21:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2120(8151796) done Nov 04 21:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_855(8127969) WEXITSTATUS 0 Nov 04 21:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_855(8127969) done Nov 04 21:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2334(8152251) WEXITSTATUS 0 Nov 04 21:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2334(8152251) done Nov 04 21:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128556 WEXITSTATUS 0 Nov 04 21:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128556 done Nov 04 21:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2917(8128601) WEXITSTATUS 1 Nov 04 21:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2917(8128601) done Nov 04 21:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2356(8152400) WEXITSTATUS 0 Nov 04 21:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2356(8152400) done Nov 04 21:43:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2081(8151757) WEXITSTATUS 0 Nov 04 21:43:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2081(8151757) done Nov 04 21:43:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4696(8156468) WEXITSTATUS 1 Nov 04 21:43:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4696(8156468) done Nov 04 21:43:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1164(8133201) WEXITSTATUS 0 Nov 04 21:43:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1164(8133201) done Nov 04 21:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4568(8156025) WEXITSTATUS 1 Nov 04 21:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4568(8156025) done Nov 04 21:43:52 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2899(8128583) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4923(8158110) WEXITSTATUS 1 Nov 04 21:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4923(8158110) done Nov 04 21:43:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2899(8128583) Nov 04 21:43:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_1(8158200) in day on r914u05n03 Nov 04 21:43:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158181 in day on r906u41n03 Nov 04 21:43:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3515(8158201) in gpu on r903u34n01 Nov 04 21:43:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3516(8158202) in gpu on r903u40n01 Nov 04 21:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4853(8157878) WEXITSTATUS 1 Nov 04 21:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4853(8157878) done Nov 04 21:44:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157470 in scavenge on r815u03n08 Nov 04 21:44:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157471 in scavenge on r815u03n08 Nov 04 21:44:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3512(8158182) WEXITSTATUS 0 Nov 04 21:44:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3512(8158182) done Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4953(8158203) in scavenge on r908u39n03 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4954(8158204) in scavenge on r917u07n04 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4955(8158205) in scavenge on r801u04n03 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4956(8158206) in scavenge on r801u04n03 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4957(8158207) in scavenge on r801u04n04 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4958(8158208) in scavenge on r801u09n03 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4959(8158209) in scavenge on r801u09n03 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4960(8158210) in scavenge on r802u02n03 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4961(8158211) in scavenge on r802u06n03 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4962(8158212) in scavenge on r802u23n02 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4963(8158213) in scavenge on r805u25n02 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4964(8158214) in scavenge on r909u07n04 Nov 04 21:44:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4965(8158215) in scavenge on r909u10n01 Nov 04 21:44:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_274(7572442) in scavenge on r801u04n04 Nov 04 21:44:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4732(8156861) WEXITSTATUS 1 Nov 04 21:44:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4732(8156861) done Nov 04 21:44:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155971 WEXITSTATUS 0 Nov 04 21:44:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155971 done Nov 04 21:44:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Unspecified error Nov 04 21:44:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:900 Nov 04 21:44:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 04 21:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1062(8129774) WEXITSTATUS 0 Nov 04 21:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1062(8129774) done Nov 04 21:44:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158216 InitPrio=35966 usec=5941 Nov 04 21:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2329(8152246) WEXITSTATUS 0 Nov 04 21:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2329(8152246) done Nov 04 21:44:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1804(8158217) in scavenge on r801u04n03 Nov 04 21:44:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1805(8158218) in scavenge on r801u09n03 Nov 04 21:44:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1806(8158219) in scavenge on r912u09n01 Nov 04 21:44:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4795(8157691) WEXITSTATUS 1 Nov 04 21:44:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4795(8157691) done Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1634(8150724) WEXITSTATUS 0 Nov 04 21:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1634(8150724) done Nov 04 21:44:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155973 WEXITSTATUS 0 Nov 04 21:44:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155973 done Nov 04 21:44:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4641(8156245) WEXITSTATUS 1 Nov 04 21:44:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4641(8156245) done Nov 04 21:45:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1824(8151150) WEXITSTATUS 0 Nov 04 21:45:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1824(8151150) done Nov 04 21:45:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155978 WEXITSTATUS 0 Nov 04 21:45:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155978 done Nov 04 21:45:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157472 NodeList=r815u09n12 #CPUs=12 Partition=scavenge Nov 04 21:45:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157473 NodeList=r815u23n09 #CPUs=12 Partition=scavenge Nov 04 21:45:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8143793 Nov 04 21:45:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_19(8146698) Nov 04 21:45:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3499(8157982) WEXITSTATUS 0 Nov 04 21:45:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3499(8157982) done Nov 04 21:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1547(8150526) WEXITSTATUS 0 Nov 04 21:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1547(8150526) done Nov 04 21:45:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155982 WEXITSTATUS 0 Nov 04 21:45:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155982 done Nov 04 21:45:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3510(8158141) WEXITSTATUS 0 Nov 04 21:45:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3510(8158141) done Nov 04 21:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128553 WEXITSTATUS 0 Nov 04 21:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128553 done Nov 04 21:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4792(8157688) WEXITSTATUS 1 Nov 04 21:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4792(8157688) done Nov 04 21:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157474 NodeList=r816u15n10 #CPUs=12 Partition=scavenge Nov 04 21:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157475 NodeList=r816u29n08 #CPUs=12 Partition=scavenge Nov 04 21:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155980 WEXITSTATUS 0 Nov 04 21:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155980 done Nov 04 21:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2221(8152013) WEXITSTATUS 0 Nov 04 21:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2221(8152013) done Nov 04 21:46:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155981 WEXITSTATUS 0 Nov 04 21:46:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155981 done Nov 04 21:46:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3503(8158070) WEXITSTATUS 0 Nov 04 21:46:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3503(8158070) done Nov 04 21:46:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1600(8150579) WEXITSTATUS 0 Nov 04 21:46:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1600(8150579) done Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:46:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4904(8158054) WEXITSTATUS 1 Nov 04 21:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4904(8158054) done Nov 04 21:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_838(8127924) WEXITSTATUS 0 Nov 04 21:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_838(8127924) done Nov 04 21:46:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3497(8157969) WEXITSTATUS 0 Nov 04 21:46:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3497(8157969) done Nov 04 21:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1049(8129021) WEXITSTATUS 0 Nov 04 21:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1049(8129021) done Nov 04 21:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3478(8157777) WEXITSTATUS 0 Nov 04 21:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3478(8157777) done Nov 04 21:46:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155979 WEXITSTATUS 0 Nov 04 21:46:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155979 done Nov 04 21:46:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1187(7572154) Nov 04 21:46:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1194(7572161) Nov 04 21:46:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_111(7572278) Nov 04 21:46:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_117(7572284) Nov 04 21:46:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_136(7572303) Nov 04 21:47:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155989 WEXITSTATUS 0 Nov 04 21:47:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155989 done Nov 04 21:47:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_2(8158220) in day on r909u04n01 Nov 04 21:47:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_3(8158221) in day on r919u36n03 Nov 04 21:47:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158216 in day on r906u41n03 Nov 04 21:47:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3517(8158222) in gpu on r903u34n01 Nov 04 21:47:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3518(8158223) in gpu on r808u08n01 Nov 04 21:47:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3519(8158224) in gpu on r808u14n01 Nov 04 21:47:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3520(8158225) in gpu on r808u17n01 Nov 04 21:47:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3521(8158226) in gpu on r4513u20n01 Nov 04 21:47:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3522(8158227) in gpu on r4513u23n01 Nov 04 21:47:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4781(8157463) WEXITSTATUS 1 Nov 04 21:47:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4781(8157463) done Nov 04 21:47:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157476 NodeList=r816u15n11 #CPUs=12 Partition=scavenge Nov 04 21:47:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157477 NodeList=r816u23n08 #CPUs=12 Partition=scavenge Nov 04 21:47:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157478 NodeList=r816u23n11 #CPUs=12 Partition=scavenge Nov 04 21:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 21:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157479 in scavenge on r815u35n02 Nov 04 21:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157480 in scavenge on r815u35n02 Nov 04 21:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_61(8156414) WEXITSTATUS 0 Nov 04 21:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_61(8156414) done Nov 04 21:47:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1752(8124564) Nov 04 21:47:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4249(8153125) WEXITSTATUS 1 Nov 04 21:47:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4249(8153125) done Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4966(8158228) in scavenge on r907u01n02 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4967(8158229) in scavenge on r907u01n02 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4968(8158230) in scavenge on r907u01n03 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4969(8158231) in scavenge on r907u01n03 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4970(8158232) in scavenge on r907u01n04 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4971(8158233) in scavenge on r907u01n04 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4972(8158234) in scavenge on r907u05n02 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4973(8158235) in scavenge on r907u05n02 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4974(8158236) in scavenge on r911u05n01 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4975(8158237) in scavenge on r915u32n01 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4976(8158238) in scavenge on r915u35n02 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4977(8158239) in scavenge on r816u35n07 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4978(8158240) in scavenge on r805u27n04 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4979(8158241) in scavenge on r918u32n03 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4980(8158242) in scavenge on r918u32n03 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4981(8158243) in scavenge on r918u32n03 Nov 04 21:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4982(8158244) in scavenge on r919u05n03 Nov 04 21:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3517(8158222) WEXITSTATUS 0 Nov 04 21:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3517(8158222) done Nov 04 21:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4810(8157767) WEXITSTATUS 1 Nov 04 21:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4810(8157767) done Nov 04 21:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3516(8158202) WEXITSTATUS 0 Nov 04 21:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3516(8158202) done Nov 04 21:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133248 WEXITSTATUS 0 Nov 04 21:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133248 done Nov 04 21:47:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1807(8158245) in scavenge on r906u13n01 Nov 04 21:47:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1808(8158246) in scavenge on r906u13n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:47:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3515(8158201) WEXITSTATUS 0 Nov 04 21:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3515(8158201) done Nov 04 21:47:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_20(8146807) Nov 04 21:47:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_21(8146808) Nov 04 21:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4888(8157999) WEXITSTATUS 1 Nov 04 21:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4888(8157999) done Nov 04 21:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1946(8151384) WEXITSTATUS 0 Nov 04 21:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1946(8151384) done Nov 04 21:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4927(8158114) WEXITSTATUS 1 Nov 04 21:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4927(8158114) done Nov 04 21:48:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_4(8158247) NodeList=r815u29n06 #CPUs=8 Partition=day Nov 04 21:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4134(8149609) WEXITSTATUS 1 Nov 04 21:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4134(8149609) done Nov 04 21:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4948(8158193) WEXITSTATUS 1 Nov 04 21:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4948(8158193) done Nov 04 21:48:23 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1754(8124567) Nov 04 21:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2047(8151607) WEXITSTATUS 0 Nov 04 21:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2047(8151607) done Nov 04 21:48:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_49(8156402) WEXITSTATUS 0 Nov 04 21:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_49(8156402) done Nov 04 21:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_59(8156412) WEXITSTATUS 0 Nov 04 21:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_59(8156412) done Nov 04 21:48:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1231(8135104) WEXITSTATUS 0 Nov 04 21:48:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1231(8135104) done Nov 04 21:48:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158248 InitPrio=34228 usec=4822 Nov 04 21:48:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158249 InitPrio=59389 usec=6559 Nov 04 21:48:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1738(8150949) WEXITSTATUS 0 Nov 04 21:48:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1738(8150949) done Nov 04 21:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1843(8151169) WEXITSTATUS 0 Nov 04 21:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1843(8151169) done Nov 04 21:48:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2197(8151989) WEXITSTATUS 0 Nov 04 21:48:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2197(8151989) done Nov 04 21:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8149447 WEXITSTATUS 0 Nov 04 21:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8149447 done Nov 04 21:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3513(8158183) WEXITSTATUS 0 Nov 04 21:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3513(8158183) done Nov 04 21:49:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155990 WEXITSTATUS 0 Nov 04 21:49:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155990 done Nov 04 21:49:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_42(8156395) WEXITSTATUS 0 Nov 04 21:49:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_42(8156395) done Nov 04 21:49:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158249 NodeList=r909u10n04 #CPUs=8 Partition=day Nov 04 21:49:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_5(8158250) NodeList=r919u09n04 #CPUs=8 Partition=day Nov 04 21:49:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157481 NodeList=r815u15n12 #CPUs=12 Partition=scavenge Nov 04 21:49:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157482 NodeList=r816u15n10 #CPUs=12 Partition=scavenge Nov 04 21:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4234(8152487) WEXITSTATUS 1 Nov 04 21:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4234(8152487) done Nov 04 21:49:23 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3866(8144931) has been requeued to reclaim resources for JobId=Invalid Nov 04 21:49:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3866(8144931) Nov 04 21:49:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158248 NodeList=r801u06n01 #CPUs=4 Partition=education Nov 04 21:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_56(8156409) WEXITSTATUS 0 Nov 04 21:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_56(8156409) done Nov 04 21:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_58(8156411) WEXITSTATUS 0 Nov 04 21:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_58(8156411) done Nov 04 21:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_52(8156405) WEXITSTATUS 0 Nov 04 21:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_52(8156405) done Nov 04 21:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_51(8156404) WEXITSTATUS 0 Nov 04 21:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_51(8156404) done Nov 04 21:49:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_65(8156418) WEXITSTATUS 0 Nov 04 21:49:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_65(8156418) done Nov 04 21:49:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:49:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 21:49:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_68(8156421) WEXITSTATUS 0 Nov 04 21:49:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_68(8156421) done Nov 04 21:49:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_50(8156403) WEXITSTATUS 0 Nov 04 21:49:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_50(8156403) done Nov 04 21:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1849(8151175) WEXITSTATUS 0 Nov 04 21:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1849(8151175) done Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:49:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:49:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4491(8155864) WEXITSTATUS 1 Nov 04 21:49:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4491(8155864) done Nov 04 21:49:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4953(8158203) WEXITSTATUS 1 Nov 04 21:49:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4953(8158203) done Nov 04 21:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4747(8157097) WEXITSTATUS 1 Nov 04 21:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4747(8157097) done Nov 04 21:50:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_66(8156419) WEXITSTATUS 0 Nov 04 21:50:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_66(8156419) done Nov 04 21:50:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155983 WEXITSTATUS 0 Nov 04 21:50:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155983 done Nov 04 21:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2155(8151831) WEXITSTATUS 0 Nov 04 21:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2155(8151831) done Nov 04 21:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4820(8157786) WEXITSTATUS 1 Nov 04 21:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4820(8157786) done Nov 04 21:50:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157483 NodeList=r816u29n08 #CPUs=12 Partition=scavenge Nov 04 21:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4378(8155205) WEXITSTATUS 1 Nov 04 21:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4378(8155205) done Nov 04 21:50:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158098 WEXITSTATUS 0 Nov 04 21:50:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158098 done Nov 04 21:50:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_57(8156410) WEXITSTATUS 0 Nov 04 21:50:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_57(8156410) done Nov 04 21:50:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_63(8156416) WEXITSTATUS 0 Nov 04 21:50:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_63(8156416) done Nov 04 21:50:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_6(8158251) in day on r909u01n04 Nov 04 21:50:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3523(8158252) in gpu on r4513u32n01 Nov 04 21:50:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3524(8158253) in gpu on r903u34n01 Nov 04 21:50:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3525(8158254) in gpu on r903u34n01 Nov 04 21:50:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3526(8158255) in gpu on r903u34n01 Nov 04 21:50:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157484 in scavenge on r815u03n10 Nov 04 21:50:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157485 in scavenge on r815u15n04 Nov 04 21:50:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157486 in scavenge on r815u15n04 Nov 04 21:50:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4966(8158228) WEXITSTATUS 1 Nov 04 21:50:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4966(8158228) done Nov 04 21:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2336(8152253) WEXITSTATUS 0 Nov 04 21:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2336(8152253) done Nov 04 21:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3498(8157981) WEXITSTATUS 0 Nov 04 21:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3498(8157981) done Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4983(8158256) in scavenge on r908u39n03 Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4984(8158257) in scavenge on r916u04n02 Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4985(8158258) in scavenge on r802u06n01 Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4986(8158259) in scavenge on r802u06n02 Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4987(8158260) in scavenge on r802u11n01 Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4988(8158261) in scavenge on r806u04n03 Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4989(8158262) in scavenge on r906u41n03 Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4990(8158263) in scavenge on r909u01n04 Nov 04 21:50:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4991(8158264) in scavenge on r806u41n01 Nov 04 21:50:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_67(8156420) WEXITSTATUS 0 Nov 04 21:50:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_67(8156420) done Nov 04 21:50:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1636(8150726) WEXITSTATUS 0 Nov 04 21:50:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1636(8150726) done Nov 04 21:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_62(8156415) WEXITSTATUS 0 Nov 04 21:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_62(8156415) done Nov 04 21:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4724(8156735) WEXITSTATUS 1 Nov 04 21:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4724(8156735) done Nov 04 21:50:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_53(8156406) WEXITSTATUS 0 Nov 04 21:50:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_53(8156406) done Nov 04 21:50:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_60(8156413) WEXITSTATUS 0 Nov 04 21:50:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_60(8156413) done Nov 04 21:50:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3526(8158255) WEXITSTATUS 0 Nov 04 21:50:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3526(8158255) done Nov 04 21:50:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155996 WEXITSTATUS 0 Nov 04 21:50:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155996 done Nov 04 21:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4832(8157811) WEXITSTATUS 1 Nov 04 21:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4832(8157811) done Nov 04 21:50:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_22(8147020) Nov 04 21:50:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_23(8147021) Nov 04 21:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1235(8135118) WEXITSTATUS 0 Nov 04 21:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1235(8135118) done Nov 04 21:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_54(8156407) WEXITSTATUS 0 Nov 04 21:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_54(8156407) done Nov 04 21:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_64(8156417) WEXITSTATUS 0 Nov 04 21:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_64(8156417) done Nov 04 21:51:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_55(8156408) WEXITSTATUS 0 Nov 04 21:51:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_55(8156408) done Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:51:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 04 21:51:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 04 21:51:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 04 21:51:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_7(8158265) NodeList=r919u32n04 #CPUs=8 Partition=day Nov 04 21:51:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_8(8158266) NodeList=r816u09n07 #CPUs=8 Partition=day Nov 04 21:51:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157487 NodeList=r816u15n12 #CPUs=12 Partition=scavenge Nov 04 21:51:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_48(8156401) WEXITSTATUS 0 Nov 04 21:51:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_48(8156401) done Nov 04 21:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4951(8158196) WEXITSTATUS 1 Nov 04 21:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4951(8158196) done Nov 04 21:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4572(8156081) WEXITSTATUS 1 Nov 04 21:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4572(8156081) done Nov 04 21:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156001 WEXITSTATUS 0 Nov 04 21:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156001 done Nov 04 21:51:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4920(8158083) WEXITSTATUS 1 Nov 04 21:51:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4920(8158083) done Nov 04 21:51:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3523(8158252) WEXITSTATUS 0 Nov 04 21:51:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3523(8158252) done Nov 04 21:51:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_445(7571410) Nov 04 21:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1060(8129772) WEXITSTATUS 0 Nov 04 21:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1060(8129772) done Nov 04 21:52:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1811(8151022) WEXITSTATUS 0 Nov 04 21:52:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1811(8151022) done Nov 04 21:52:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1250(8135152) WEXITSTATUS 0 Nov 04 21:52:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1250(8135152) done Nov 04 21:52:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156006 WEXITSTATUS 0 Nov 04 21:52:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156006 done Nov 04 21:52:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1732(8150943) WEXITSTATUS 0 Nov 04 21:52:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1732(8150943) done Nov 04 21:52:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4674(8156336) WEXITSTATUS 1 Nov 04 21:52:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4674(8156336) done Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:53:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:53:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_9(8158267) NodeList=r919u32n04 #CPUs=8 Partition=day Nov 04 21:53:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157488 NodeList=r816u23n10 #CPUs=12 Partition=scavenge Nov 04 21:53:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_2(8156355) WEXITSTATUS 0 Nov 04 21:53:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_2(8156355) done Nov 04 21:53:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1760(8124589) Nov 04 21:53:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3527(8158268) in gpu on r808u08n01 Nov 04 21:53:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3528(8158269) in gpu on r4513u32n01 Nov 04 21:53:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3529(8158270) in gpu on r903u34n01 Nov 04 21:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n03 Nov 04 21:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157489 in scavenge on r815u09n01 Nov 04 21:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157490 in scavenge on r815u09n01 Nov 04 21:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157491 in scavenge on r815u15n08 Nov 04 21:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_108(8156485) WEXITSTATUS 0 Nov 04 21:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_108(8156485) done Nov 04 21:53:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4090(8148570) WEXITSTATUS 1 Nov 04 21:53:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4090(8148570) done Nov 04 21:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_105(8156482) WEXITSTATUS 0 Nov 04 21:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_105(8156482) done Nov 04 21:53:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4992(8158271) in scavenge on r907u01n02 Nov 04 21:53:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4993(8158272) in scavenge on r801u06n03 Nov 04 21:53:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4994(8158273) in scavenge on r808u08n01 Nov 04 21:53:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4995(8158274) in scavenge on r806u14n04 Nov 04 21:53:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4996(8158275) in scavenge on r913u05n03 Nov 04 21:53:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4997(8158276) in scavenge on r919u36n01 Nov 04 21:53:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4998(8158277) in scavenge on r815u23n02 Nov 04 21:53:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144452 Nov 04 21:53:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_24(8147205) Nov 04 21:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_110(8156487) WEXITSTATUS 0 Nov 04 21:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_110(8156487) done Nov 04 21:54:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1809(8158278) in scavenge on r906u13n01 Nov 04 21:54:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 21:54:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_10(8158279) NodeList=r919u36n04 #CPUs=8 Partition=day Nov 04 21:54:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_165(8156542) WEXITSTATUS 0 Nov 04 21:54:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_165(8156542) done Nov 04 21:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4858(8157943) WEXITSTATUS 1 Nov 04 21:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4858(8157943) done Nov 04 21:54:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4759(8157330) WEXITSTATUS 1 Nov 04 21:54:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4759(8157330) done Nov 04 21:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_4(8156357) WEXITSTATUS 0 Nov 04 21:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_4(8156357) done Nov 04 21:54:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_81(7572248) Nov 04 21:54:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1763(8124594) Nov 04 21:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_18(8156371) WEXITSTATUS 0 Nov 04 21:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_18(8156371) done Nov 04 21:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_38(8156391) WEXITSTATUS 0 Nov 04 21:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_38(8156391) done Nov 04 21:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_77(8156430) WEXITSTATUS 0 Nov 04 21:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_77(8156430) done Nov 04 21:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3660(8138023) WEXITSTATUS 1 Nov 04 21:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3660(8138023) done Nov 04 21:54:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_76(8156429) WEXITSTATUS 0 Nov 04 21:54:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_76(8156429) done Nov 04 21:54:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_941(8128227) WEXITSTATUS 0 Nov 04 21:54:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_941(8128227) done Nov 04 21:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2110(8151786) WEXITSTATUS 0 Nov 04 21:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2110(8151786) done Nov 04 21:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_69(8156422) WEXITSTATUS 0 Nov 04 21:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_69(8156422) done Nov 04 21:54:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 21:54:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_19(8156372) WEXITSTATUS 0 Nov 04 21:54:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_19(8156372) done Nov 04 21:55:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2324(8152241) WEXITSTATUS 0 Nov 04 21:55:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2324(8152241) done Nov 04 21:55:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3476(8135398) WEXITSTATUS 1 Nov 04 21:55:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3476(8135398) done Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:55:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:55:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_11(8158280) NodeList=r909u10n02 #CPUs=8 Partition=day Nov 04 21:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_104(8156481) WEXITSTATUS 0 Nov 04 21:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_104(8156481) done Nov 04 21:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_16(8156369) WEXITSTATUS 0 Nov 04 21:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_16(8156369) done Nov 04 21:55:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_174(8156551) WEXITSTATUS 0 Nov 04 21:55:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_174(8156551) done Nov 04 21:55:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_107(8156484) WEXITSTATUS 0 Nov 04 21:55:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_107(8156484) done Nov 04 21:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3524(8158253) WEXITSTATUS 0 Nov 04 21:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3524(8158253) done Nov 04 21:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_5(8156358) WEXITSTATUS 0 Nov 04 21:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_5(8156358) done Nov 04 21:55:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4676(8156338) WEXITSTATUS 1 Nov 04 21:55:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4676(8156338) done Nov 04 21:55:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_44(8156397) WEXITSTATUS 0 Nov 04 21:55:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_44(8156397) done Nov 04 21:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_176(8156553) WEXITSTATUS 0 Nov 04 21:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_176(8156553) done Nov 04 21:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_8(8156361) WEXITSTATUS 0 Nov 04 21:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_8(8156361) done Nov 04 21:55:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_20(8156373) WEXITSTATUS 0 Nov 04 21:55:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_20(8156373) done Nov 04 21:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_40(8156393) WEXITSTATUS 0 Nov 04 21:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_40(8156393) done Nov 04 21:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_14(8156367) WEXITSTATUS 0 Nov 04 21:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_14(8156367) done Nov 04 21:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_41(8156394) WEXITSTATUS 0 Nov 04 21:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_41(8156394) done Nov 04 21:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_73(8156426) WEXITSTATUS 0 Nov 04 21:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_73(8156426) done Nov 04 21:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4921(8158108) WEXITSTATUS 1 Nov 04 21:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4921(8158108) done Nov 04 21:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_32(8156385) WEXITSTATUS 0 Nov 04 21:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_32(8156385) done Nov 04 21:55:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144453 Nov 04 21:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_72(8156425) WEXITSTATUS 0 Nov 04 21:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_72(8156425) done Nov 04 21:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_21(8156374) WEXITSTATUS 0 Nov 04 21:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_21(8156374) done Nov 04 21:56:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155995 WEXITSTATUS 0 Nov 04 21:56:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8155995 done Nov 04 21:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1249(8135151) WEXITSTATUS 0 Nov 04 21:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1249(8135151) done Nov 04 21:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_45(8156398) WEXITSTATUS 0 Nov 04 21:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_45(8156398) done Nov 04 21:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4436(8155677) WEXITSTATUS 1 Nov 04 21:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4436(8155677) done Nov 04 21:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4979(8158241) WEXITSTATUS 1 Nov 04 21:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4979(8158241) done Nov 04 21:56:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_46(8156399) WEXITSTATUS 0 Nov 04 21:56:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_46(8156399) done Nov 04 21:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_175(8156552) WEXITSTATUS 0 Nov 04 21:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_175(8156552) done Nov 04 21:56:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_12(8158281) NodeList=r919u09n02 #CPUs=8 Partition=day Nov 04 21:56:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157492 NodeList=r816u29n10 #CPUs=12 Partition=scavenge Nov 04 21:56:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4993(8158272) WEXITSTATUS 1 Nov 04 21:56:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4993(8158272) done Nov 04 21:56:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4938(8158147) WEXITSTATUS 1 Nov 04 21:56:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4938(8158147) done Nov 04 21:56:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_17(8156370) WEXITSTATUS 0 Nov 04 21:56:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_17(8156370) done Nov 04 21:56:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_101(8156478) WEXITSTATUS 0 Nov 04 21:56:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_101(8156478) done Nov 04 21:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_27(8156380) WEXITSTATUS 0 Nov 04 21:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_27(8156380) done Nov 04 21:56:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 04 21:56:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 04 21:56:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 04 21:56:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 04 21:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_10(8156363) WEXITSTATUS 0 Nov 04 21:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_10(8156363) done Nov 04 21:56:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_15(8156368) WEXITSTATUS 0 Nov 04 21:56:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_15(8156368) done Nov 04 21:56:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_96(8156449) WEXITSTATUS 0 Nov 04 21:56:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_96(8156449) done Nov 04 21:56:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_7(8156360) WEXITSTATUS 0 Nov 04 21:56:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_7(8156360) done Nov 04 21:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_178(8156555) WEXITSTATUS 0 Nov 04 21:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_178(8156555) done Nov 04 21:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_109(8156486) WEXITSTATUS 0 Nov 04 21:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_109(8156486) done Nov 04 21:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_70(8156423) WEXITSTATUS 0 Nov 04 21:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_70(8156423) done Nov 04 21:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_3(8156356) WEXITSTATUS 0 Nov 04 21:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_3(8156356) done Nov 04 21:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_13(8156366) WEXITSTATUS 0 Nov 04 21:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_13(8156366) done Nov 04 21:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4774(8157456) WEXITSTATUS 1 Nov 04 21:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4774(8157456) done Nov 04 21:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_29(8156382) WEXITSTATUS 0 Nov 04 21:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_29(8156382) done Nov 04 21:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_11(8156364) WEXITSTATUS 0 Nov 04 21:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_11(8156364) done Nov 04 21:56:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_97(8156450) WEXITSTATUS 0 Nov 04 21:56:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_97(8156450) done Nov 04 21:56:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3530(8158282) in gpu on r903u34n01 Nov 04 21:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_151(8156528) WEXITSTATUS 0 Nov 04 21:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_151(8156528) done Nov 04 21:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_22(8156375) WEXITSTATUS 0 Nov 04 21:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_22(8156375) done Nov 04 21:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4697(8156469) WEXITSTATUS 1 Nov 04 21:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4697(8156469) done Nov 04 21:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_93(8156446) WEXITSTATUS 0 Nov 04 21:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_93(8156446) done Nov 04 21:56:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_142(7572310) Nov 04 21:56:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_200(7572368) Nov 04 21:56:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_25(8147401) Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4999(8158283) in scavenge on r801u30n01 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5000(8158284) in scavenge on r802u32n02 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5001(8158285) in scavenge on r917u29n02 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5002(8158286) in scavenge on r801u06n03 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5003(8158287) in scavenge on r801u27n02 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5004(8158288) in scavenge on r807u22n01 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5005(8158289) in scavenge on r902u04n01 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5006(8158290) in scavenge on r805u04n02 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5007(8158291) in scavenge on r906u41n01 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5008(8158292) in scavenge on r904u04n03 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5009(8158293) in scavenge on r918u32n03 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5010(8158294) in scavenge on r919u01n04 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5011(8158295) in scavenge on r919u36n04 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5012(8158296) in scavenge on r919u36n04 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_6(8156359) WEXITSTATUS 0 Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_6(8156359) done Nov 04 21:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8158297 NodeList=(null) usec=19220 Nov 04 21:57:00 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158297 NodeList=r806u23n03 #CPUs=4 Partition=devel Nov 04 21:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_43(8156396) WEXITSTATUS 0 Nov 04 21:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_43(8156396) done Nov 04 21:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4827(8157806) WEXITSTATUS 1 Nov 04 21:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4827(8157806) done Nov 04 21:57:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_192(8156569) WEXITSTATUS 0 Nov 04 21:57:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_192(8156569) done Nov 04 21:57:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_149(8156526) WEXITSTATUS 0 Nov 04 21:57:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_149(8156526) done Nov 04 21:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_13(8158298) NodeList=r919u36n02 #CPUs=8 Partition=day Nov 04 21:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157493 NodeList=r816u23n09 #CPUs=12 Partition=scavenge Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:57:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:57:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_156(8156533) WEXITSTATUS 0 Nov 04 21:57:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_156(8156533) done Nov 04 21:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_33(8156386) WEXITSTATUS 0 Nov 04 21:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_33(8156386) done Nov 04 21:57:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1770(8124611) Nov 04 21:57:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1787(8124628) Nov 04 21:57:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1790(8124631) Nov 04 21:57:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_160(8156537) WEXITSTATUS 0 Nov 04 21:57:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_160(8156537) done Nov 04 21:57:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4618(8156202) WEXITSTATUS 1 Nov 04 21:57:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4618(8156202) done Nov 04 21:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_147(8156524) WEXITSTATUS 0 Nov 04 21:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_147(8156524) done Nov 04 21:57:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_126(8156503) WEXITSTATUS 0 Nov 04 21:57:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_126(8156503) done Nov 04 21:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_25(8156378) WEXITSTATUS 0 Nov 04 21:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_25(8156378) done Nov 04 21:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4836(8157833) WEXITSTATUS 1 Nov 04 21:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4836(8157833) done Nov 04 21:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_12(8156365) WEXITSTATUS 0 Nov 04 21:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_12(8156365) done Nov 04 21:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_132(8156509) WEXITSTATUS 0 Nov 04 21:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_132(8156509) done Nov 04 21:57:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_9(8156362) WEXITSTATUS 0 Nov 04 21:57:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_9(8156362) done Nov 04 21:57:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_36(8156389) WEXITSTATUS 0 Nov 04 21:57:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_36(8156389) done Nov 04 21:57:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_24(8156377) WEXITSTATUS 0 Nov 04 21:57:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_24(8156377) done Nov 04 21:57:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_164(8156541) WEXITSTATUS 0 Nov 04 21:57:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_164(8156541) done Nov 04 21:57:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_99(8156452) WEXITSTATUS 0 Nov 04 21:57:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_99(8156452) done Nov 04 21:57:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_161(8156538) WEXITSTATUS 0 Nov 04 21:57:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_161(8156538) done Nov 04 21:57:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_23(8156376) WEXITSTATUS 0 Nov 04 21:57:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_23(8156376) done Nov 04 21:57:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_95(8156448) WEXITSTATUS 0 Nov 04 21:57:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_95(8156448) done Nov 04 21:57:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_275(8156683) WEXITSTATUS 0 Nov 04 21:57:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_275(8156683) done Nov 04 21:57:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_162(8156539) WEXITSTATUS 0 Nov 04 21:57:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_162(8156539) done Nov 04 21:58:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_293(8156701) WEXITSTATUS 0 Nov 04 21:58:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_293(8156701) done Nov 04 21:58:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_148(8156525) WEXITSTATUS 0 Nov 04 21:58:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_148(8156525) done Nov 04 21:58:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133250 WEXITSTATUS 0 Nov 04 21:58:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133250 done Nov 04 21:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1207(8134438) WEXITSTATUS 0 Nov 04 21:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1207(8134438) done Nov 04 21:58:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_102(8156479) WEXITSTATUS 0 Nov 04 21:58:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_102(8156479) done Nov 04 21:58:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_188(8156565) WEXITSTATUS 0 Nov 04 21:58:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_188(8156565) done Nov 04 21:58:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158125_14(8158299) NodeList=r914u05n04 #CPUs=8 Partition=day Nov 04 21:58:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157494 NodeList=r816u03n12 #CPUs=12 Partition=scavenge Nov 04 21:58:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157495 NodeList=r816u35n05 #CPUs=12 Partition=scavenge Nov 04 21:58:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157496 NodeList=r816u35n05 #CPUs=12 Partition=scavenge Nov 04 21:58:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4883(8157994) WEXITSTATUS 1 Nov 04 21:58:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4883(8157994) done Nov 04 21:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907435 WEXITSTATUS 0 Nov 04 21:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907435 done Nov 04 21:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_158(8156535) WEXITSTATUS 0 Nov 04 21:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_158(8156535) done Nov 04 21:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_127(8156504) WEXITSTATUS 0 Nov 04 21:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_127(8156504) done Nov 04 21:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_135(8156512) WEXITSTATUS 0 Nov 04 21:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_135(8156512) done Nov 04 21:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_124(8156501) WEXITSTATUS 0 Nov 04 21:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_124(8156501) done Nov 04 21:58:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_131(8156508) WEXITSTATUS 0 Nov 04 21:58:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_131(8156508) done Nov 04 21:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_94(8156447) WEXITSTATUS 0 Nov 04 21:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_94(8156447) done Nov 04 21:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156011 WEXITSTATUS 0 Nov 04 21:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156011 done Nov 04 21:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_292(8156700) WEXITSTATUS 0 Nov 04 21:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_292(8156700) done Nov 04 21:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5001(8158285) WEXITSTATUS 1 Nov 04 21:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5001(8158285) done Nov 04 21:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25892: QOSMaxMemoryPerUser Nov 04 21:58:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 21:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_157(8156534) WEXITSTATUS 0 Nov 04 21:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_157(8156534) done Nov 04 21:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_37(8156390) WEXITSTATUS 0 Nov 04 21:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_37(8156390) done Nov 04 21:58:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_179(8156556) WEXITSTATUS 0 Nov 04 21:58:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_179(8156556) done Nov 04 21:58:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_92(8156445) WEXITSTATUS 0 Nov 04 21:58:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_92(8156445) done Nov 04 21:58:40 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25892: QOSMaxMemoryPerUser Nov 04 21:58:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 21:58:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_100(8156477) WEXITSTATUS 0 Nov 04 21:58:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_100(8156477) done Nov 04 21:58:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_159(8156536) WEXITSTATUS 0 Nov 04 21:58:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_159(8156536) done Nov 04 21:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_185(8156562) WEXITSTATUS 0 Nov 04 21:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_185(8156562) done Nov 04 21:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_154(8156531) WEXITSTATUS 0 Nov 04 21:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_154(8156531) done Nov 04 21:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_106(8156483) WEXITSTATUS 0 Nov 04 21:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_106(8156483) done Nov 04 21:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_273(8156681) WEXITSTATUS 0 Nov 04 21:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_273(8156681) done Nov 04 21:58:51 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25892: QOSMaxMemoryPerUser Nov 04 21:58:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 21:58:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_130(8156507) WEXITSTATUS 0 Nov 04 21:58:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_130(8156507) done Nov 04 21:58:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_398(8156836) WEXITSTATUS 0 Nov 04 21:58:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_398(8156836) done Nov 04 21:58:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_133(8156510) WEXITSTATUS 0 Nov 04 21:58:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_133(8156510) done Nov 04 21:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133249 WEXITSTATUS 0 Nov 04 21:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133249 done Nov 04 21:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_31(8156384) WEXITSTATUS 0 Nov 04 21:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_31(8156384) done Nov 04 21:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4969(8158231) WEXITSTATUS 1 Nov 04 21:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4969(8158231) done Nov 04 21:58:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_182(8156559) WEXITSTATUS 0 Nov 04 21:58:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_182(8156559) done Nov 04 21:58:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133247 WEXITSTATUS 0 Nov 04 21:58:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133247 done Nov 04 21:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_128(8156505) WEXITSTATUS 0 Nov 04 21:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_128(8156505) done Nov 04 21:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_155(8156532) WEXITSTATUS 0 Nov 04 21:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_155(8156532) done Nov 04 21:59:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_181(8156558) WEXITSTATUS 0 Nov 04 21:59:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_181(8156558) done Nov 04 21:59:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158300 InitPrio=36240 usec=4691 Nov 04 21:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_150(8156527) WEXITSTATUS 0 Nov 04 21:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_150(8156527) done Nov 04 21:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_190(8156567) WEXITSTATUS 0 Nov 04 21:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_190(8156567) done Nov 04 21:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_189(8156566) WEXITSTATUS 0 Nov 04 21:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_189(8156566) done Nov 04 21:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_91(8156444) WEXITSTATUS 0 Nov 04 21:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_91(8156444) done Nov 04 21:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158300 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 21:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_243(8156651) WEXITSTATUS 0 Nov 04 21:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_243(8156651) done Nov 04 21:59:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1868(8151194) WEXITSTATUS 0 Nov 04 21:59:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1868(8151194) done Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 21:59:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 21:59:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158301 InitPrio=33245 usec=3974 Nov 04 21:59:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_846(8127943) WEXITSTATUS 0 Nov 04 21:59:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_846(8127943) done Nov 04 21:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4839(8157847) WEXITSTATUS 1 Nov 04 21:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4839(8157847) done Nov 04 21:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5009(8158293) WEXITSTATUS 1 Nov 04 21:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5009(8158293) done Nov 04 21:59:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_120(8156497) WEXITSTATUS 0 Nov 04 21:59:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_120(8156497) done Nov 04 21:59:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_397(8156835) WEXITSTATUS 0 Nov 04 21:59:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_397(8156835) done Nov 04 21:59:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_165(7572333) Nov 04 21:59:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_255(8156663) WEXITSTATUS 0 Nov 04 21:59:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_255(8156663) done Nov 04 21:59:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_152(8156529) WEXITSTATUS 0 Nov 04 21:59:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_152(8156529) done Nov 04 21:59:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2032(8151592) WEXITSTATUS 0 Nov 04 21:59:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2032(8151592) done Nov 04 21:59:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_186(8156563) WEXITSTATUS 0 Nov 04 21:59:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_186(8156563) done Nov 04 21:59:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_74(8156427) WEXITSTATUS 0 Nov 04 21:59:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_74(8156427) done Nov 04 21:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_123(8156500) WEXITSTATUS 0 Nov 04 21:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_123(8156500) done Nov 04 21:59:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_153(8156530) WEXITSTATUS 0 Nov 04 21:59:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_153(8156530) done Nov 04 21:59:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158302 InitPrio=26179 usec=9644 Nov 04 21:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_134(8156511) WEXITSTATUS 0 Nov 04 21:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_134(8156511) done Nov 04 21:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_118(8156495) WEXITSTATUS 0 Nov 04 21:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_118(8156495) done Nov 04 21:59:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_191(8156568) WEXITSTATUS 0 Nov 04 21:59:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_191(8156568) done Nov 04 21:59:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158302 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 21:59:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_163(8156540) WEXITSTATUS 0 Nov 04 21:59:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_163(8156540) done Nov 04 21:59:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_111(8156488) WEXITSTATUS 0 Nov 04 21:59:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_111(8156488) done Nov 04 21:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_171(8156548) WEXITSTATUS 0 Nov 04 21:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_171(8156548) done Nov 04 21:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_15(8158303) in day on r918u32n03 Nov 04 21:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158301 in day on r906u41n01 Nov 04 21:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907436 in gpu on r805u38n01 Nov 04 21:59:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_125(8156502) WEXITSTATUS 0 Nov 04 21:59:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_125(8156502) done Nov 04 21:59:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n04 Nov 04 21:59:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157497 in scavenge on r816u15n03 Nov 04 21:59:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_265(8156673) WEXITSTATUS 0 Nov 04 21:59:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_265(8156673) done Nov 04 21:59:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_26(8147561) Nov 04 21:59:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_27(8147562) Nov 04 21:59:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_28(8147563) Nov 04 21:59:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_29(8147564) Nov 04 21:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_121(8156498) WEXITSTATUS 0 Nov 04 21:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_121(8156498) done Nov 04 21:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_1(8156354) WEXITSTATUS 0 Nov 04 21:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_1(8156354) done Nov 04 21:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158018 WEXITSTATUS 0 Nov 04 21:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158018 done Nov 04 21:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_486(8156951) WEXITSTATUS 0 Nov 04 21:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_486(8156951) done Nov 04 21:59:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4755(8157216) WEXITSTATUS 1 Nov 04 21:59:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4755(8157216) done Nov 04 22:00:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1855(8151181) WEXITSTATUS 0 Nov 04 22:00:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1855(8151181) done Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5013(8158304) in scavenge on r907u01n03 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5014(8158305) in scavenge on r908u33n04 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5015(8158306) in scavenge on r802u11n02 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5016(8158307) in scavenge on r911u05n01 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5017(8158308) in scavenge on r915u38n02 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5018(8158309) in scavenge on r917u29n02 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5019(8158310) in scavenge on r805u14n04 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5020(8158311) in scavenge on r807u03n04 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5021(8158312) in scavenge on r904u07n03 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5022(8158313) in scavenge on r913u01n02 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5023(8158314) in scavenge on r914u01n04 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5024(8158315) in scavenge on r914u01n04 Nov 04 22:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5025(8158316) in scavenge on r919u36n02 Nov 04 22:00:03 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 04 22:00:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 04 22:00:03 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 04 22:00:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 04 22:00:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 04 22:00:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_268(8156676) WEXITSTATUS 0 Nov 04 22:00:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_268(8156676) done Nov 04 22:00:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_98(8156451) WEXITSTATUS 0 Nov 04 22:00:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_98(8156451) done Nov 04 22:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4961(8158211) WEXITSTATUS 1 Nov 04 22:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4961(8158211) done Nov 04 22:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_129(8156506) WEXITSTATUS 0 Nov 04 22:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_129(8156506) done Nov 04 22:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_197(8156574) WEXITSTATUS 0 Nov 04 22:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_197(8156574) done Nov 04 22:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_177(8156554) WEXITSTATUS 0 Nov 04 22:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_177(8156554) done Nov 04 22:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_184(8156561) WEXITSTATUS 0 Nov 04 22:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_184(8156561) done Nov 04 22:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_0(8156353) WEXITSTATUS 0 Nov 04 22:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_0(8156353) done Nov 04 22:00:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_187(8156564) WEXITSTATUS 0 Nov 04 22:00:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_187(8156564) done Nov 04 22:00:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_230(8156638) WEXITSTATUS 0 Nov 04 22:00:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_230(8156638) done Nov 04 22:00:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_122(8156499) WEXITSTATUS 0 Nov 04 22:00:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_122(8156499) done Nov 04 22:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4878(8157989) WEXITSTATUS 1 Nov 04 22:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4878(8157989) done Nov 04 22:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_114(8156491) WEXITSTATUS 0 Nov 04 22:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_114(8156491) done Nov 04 22:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_180(8156557) WEXITSTATUS 0 Nov 04 22:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_180(8156557) done Nov 04 22:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_290(8156698) WEXITSTATUS 0 Nov 04 22:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_290(8156698) done Nov 04 22:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_183(8156560) WEXITSTATUS 0 Nov 04 22:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_183(8156560) done Nov 04 22:00:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_277(8156685) WEXITSTATUS 0 Nov 04 22:00:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_277(8156685) done Nov 04 22:00:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_199(8156576) WEXITSTATUS 0 Nov 04 22:00:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_199(8156576) done Nov 04 22:00:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_256(8156664) WEXITSTATUS 0 Nov 04 22:00:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_256(8156664) done Nov 04 22:00:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_249(8156657) WEXITSTATUS 0 Nov 04 22:00:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_249(8156657) done Nov 04 22:00:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_269(8156677) WEXITSTATUS 0 Nov 04 22:00:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_269(8156677) done Nov 04 22:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_266(8156674) WEXITSTATUS 0 Nov 04 22:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_266(8156674) done Nov 04 22:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156009 WEXITSTATUS 0 Nov 04 22:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156009 done Nov 04 22:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_297(8156705) WEXITSTATUS 0 Nov 04 22:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_297(8156705) done Nov 04 22:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_173(8156550) WEXITSTATUS 0 Nov 04 22:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_173(8156550) done Nov 04 22:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158317 InitPrio=34228 usec=3430 Nov 04 22:00:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158317 NodeList=r801u06n01 #CPUs=4 Partition=education Nov 04 22:00:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_117(8156494) WEXITSTATUS 0 Nov 04 22:00:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_117(8156494) done Nov 04 22:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158317 WEXITSTATUS 64 Nov 04 22:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158317 done Nov 04 22:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_252(8156660) WEXITSTATUS 0 Nov 04 22:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_252(8156660) done Nov 04 22:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_251(8156659) WEXITSTATUS 0 Nov 04 22:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_251(8156659) done Nov 04 22:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158043 WEXITSTATUS 0 Nov 04 22:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158043 done Nov 04 22:00:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_284(8156692) WEXITSTATUS 0 Nov 04 22:00:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_284(8156692) done Nov 04 22:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_488(8156953) WEXITSTATUS 0 Nov 04 22:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_488(8156953) done Nov 04 22:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4428(8155669) WEXITSTATUS 1 Nov 04 22:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4428(8155669) done Nov 04 22:00:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:00:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_167(8156544) WEXITSTATUS 0 Nov 04 22:00:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_167(8156544) done Nov 04 22:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_116(8156493) WEXITSTATUS 0 Nov 04 22:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_116(8156493) done Nov 04 22:01:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_296(8156704) WEXITSTATUS 0 Nov 04 22:01:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_296(8156704) done Nov 04 22:01:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_300(8156738) WEXITSTATUS 0 Nov 04 22:01:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_300(8156738) done Nov 04 22:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_80(8156433) WEXITSTATUS 0 Nov 04 22:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_80(8156433) done Nov 04 22:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_283(8156691) WEXITSTATUS 0 Nov 04 22:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_283(8156691) done Nov 04 22:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157498 NodeList=r816u09n12 #CPUs=12 Partition=scavenge Nov 04 22:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_399(8156837) WEXITSTATUS 0 Nov 04 22:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_399(8156837) done Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:01:16 r912u23n01 slurmctld[163760]: slurmctld: build_job_queue has run for 2000000 usec, exiting with 100 of 14836 jobs tested, 1 job-partition pairs added Nov 04 22:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_949(8128276) WEXITSTATUS 0 Nov 04 22:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_949(8128276) done Nov 04 22:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_257(8156665) WEXITSTATUS 0 Nov 04 22:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_257(8156665) done Nov 04 22:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_289(8156697) WEXITSTATUS 0 Nov 04 22:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_289(8156697) done Nov 04 22:01:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_276(8156684) WEXITSTATUS 0 Nov 04 22:01:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_276(8156684) done Nov 04 22:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_196(8156573) WEXITSTATUS 0 Nov 04 22:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_196(8156573) done Nov 04 22:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_274(8156682) WEXITSTATUS 0 Nov 04 22:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_274(8156682) done Nov 04 22:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_195(8156572) WEXITSTATUS 0 Nov 04 22:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_195(8156572) done Nov 04 22:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_198(8156575) WEXITSTATUS 0 Nov 04 22:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_198(8156575) done Nov 04 22:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_450(8156915) WEXITSTATUS 0 Nov 04 22:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_450(8156915) done Nov 04 22:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_481(8156946) WEXITSTATUS 0 Nov 04 22:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_481(8156946) done Nov 04 22:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_193(8156570) WEXITSTATUS 0 Nov 04 22:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_193(8156570) done Nov 04 22:01:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_26(8156379) WEXITSTATUS 0 Nov 04 22:01:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_26(8156379) done Nov 04 22:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_291(8156699) WEXITSTATUS 0 Nov 04 22:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_291(8156699) done Nov 04 22:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_482(8156947) WEXITSTATUS 0 Nov 04 22:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_482(8156947) done Nov 04 22:01:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1802(8124658) Nov 04 22:01:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_247(8156655) WEXITSTATUS 0 Nov 04 22:01:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_247(8156655) done Nov 04 22:01:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_475(8156940) WEXITSTATUS 0 Nov 04 22:01:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_475(8156940) done Nov 04 22:01:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_248(8156656) WEXITSTATUS 0 Nov 04 22:01:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_248(8156656) done Nov 04 22:01:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_485(8156950) WEXITSTATUS 0 Nov 04 22:01:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_485(8156950) done Nov 04 22:01:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1807(8151018) WEXITSTATUS 0 Nov 04 22:01:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1807(8151018) done Nov 04 22:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_39(8156392) WEXITSTATUS 0 Nov 04 22:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_39(8156392) done Nov 04 22:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_287(8156695) WEXITSTATUS 0 Nov 04 22:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_287(8156695) done Nov 04 22:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_295(8156703) WEXITSTATUS 0 Nov 04 22:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_295(8156703) done Nov 04 22:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_113(8156490) WEXITSTATUS 0 Nov 04 22:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_113(8156490) done Nov 04 22:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_262(8156670) WEXITSTATUS 0 Nov 04 22:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_262(8156670) done Nov 04 22:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_170(8156547) WEXITSTATUS 0 Nov 04 22:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_170(8156547) done Nov 04 22:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156008 WEXITSTATUS 0 Nov 04 22:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156008 done Nov 04 22:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2273(8152190) WEXITSTATUS 0 Nov 04 22:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2273(8152190) done Nov 04 22:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156010 WEXITSTATUS 0 Nov 04 22:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156010 done Nov 04 22:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_487(8156952) WEXITSTATUS 0 Nov 04 22:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_487(8156952) done Nov 04 22:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_112(8156489) WEXITSTATUS 0 Nov 04 22:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_112(8156489) done Nov 04 22:01:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_83(8156436) WEXITSTATUS 0 Nov 04 22:01:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_83(8156436) done Nov 04 22:01:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_244(8156652) WEXITSTATUS 0 Nov 04 22:01:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_244(8156652) done Nov 04 22:01:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_35(8156388) WEXITSTATUS 0 Nov 04 22:01:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_35(8156388) done Nov 04 22:01:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_34(8156387) WEXITSTATUS 0 Nov 04 22:01:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_34(8156387) done Nov 04 22:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4825(8157804) WEXITSTATUS 1 Nov 04 22:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4825(8157804) done Nov 04 22:01:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:01:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:01:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1061(7572027) Nov 04 22:01:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_31(7572198) Nov 04 22:01:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_176(7572344) Nov 04 22:01:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_298(7572466) Nov 04 22:01:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_408(7572576) Nov 04 22:01:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_315(8114582) Nov 04 22:01:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_1039(8115308) Nov 04 22:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_168(8156545) WEXITSTATUS 0 Nov 04 22:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_168(8156545) done Nov 04 22:01:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_88(8156441) WEXITSTATUS 0 Nov 04 22:01:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_88(8156441) done Nov 04 22:01:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_229(8156637) WEXITSTATUS 0 Nov 04 22:01:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_229(8156637) done Nov 04 22:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156012 WEXITSTATUS 0 Nov 04 22:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156012 done Nov 04 22:02:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_253(8156661) WEXITSTATUS 0 Nov 04 22:02:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_253(8156661) done Nov 04 22:02:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1198(8134211) WEXITSTATUS 0 Nov 04 22:02:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1198(8134211) done Nov 04 22:02:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_254(8156662) WEXITSTATUS 0 Nov 04 22:02:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_254(8156662) done Nov 04 22:02:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_86(8156439) WEXITSTATUS 0 Nov 04 22:02:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_86(8156439) done Nov 04 22:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_84(8156437) WEXITSTATUS 0 Nov 04 22:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_84(8156437) done Nov 04 22:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_250(8156658) WEXITSTATUS 0 Nov 04 22:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_250(8156658) done Nov 04 22:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3869(8144934) WEXITSTATUS 1 Nov 04 22:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3869(8144934) done Nov 04 22:02:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_144(8156521) WEXITSTATUS 0 Nov 04 22:02:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_144(8156521) done Nov 04 22:02:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_471(8156936) WEXITSTATUS 0 Nov 04 22:02:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_471(8156936) done Nov 04 22:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_480(8156945) WEXITSTATUS 0 Nov 04 22:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_480(8156945) done Nov 04 22:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2323(8152240) WEXITSTATUS 0 Nov 04 22:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2323(8152240) done Nov 04 22:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_302(8156740) WEXITSTATUS 0 Nov 04 22:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_302(8156740) done Nov 04 22:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156007 WEXITSTATUS 0 Nov 04 22:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156007 done Nov 04 22:02:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157499 NodeList=r816u35n06 #CPUs=12 Partition=scavenge Nov 04 22:02:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_87(8156440) WEXITSTATUS 0 Nov 04 22:02:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_87(8156440) done Nov 04 22:02:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_270(8156678) WEXITSTATUS 0 Nov 04 22:02:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_270(8156678) done Nov 04 22:02:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_364(8156802) WEXITSTATUS 0 Nov 04 22:02:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_364(8156802) done Nov 04 22:02:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_141(8156518) WEXITSTATUS 0 Nov 04 22:02:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_141(8156518) done Nov 04 22:02:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_231(8156639) WEXITSTATUS 0 Nov 04 22:02:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_231(8156639) done Nov 04 22:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_288(8156696) WEXITSTATUS 0 Nov 04 22:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_288(8156696) done Nov 04 22:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_453(8156918) WEXITSTATUS 0 Nov 04 22:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_453(8156918) done Nov 04 22:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_81(8156434) WEXITSTATUS 0 Nov 04 22:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_81(8156434) done Nov 04 22:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_285(8156693) WEXITSTATUS 0 Nov 04 22:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_285(8156693) done Nov 04 22:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_468(8156933) WEXITSTATUS 0 Nov 04 22:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_468(8156933) done Nov 04 22:02:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_278(8156686) WEXITSTATUS 0 Nov 04 22:02:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_278(8156686) done Nov 04 22:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_143(8156520) WEXITSTATUS 0 Nov 04 22:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_143(8156520) done Nov 04 22:02:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_245(8156653) WEXITSTATUS 0 Nov 04 22:02:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_245(8156653) done Nov 04 22:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_294(8156702) WEXITSTATUS 0 Nov 04 22:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_294(8156702) done Nov 04 22:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_115(8156492) WEXITSTATUS 0 Nov 04 22:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_115(8156492) done Nov 04 22:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_279(8156687) WEXITSTATUS 0 Nov 04 22:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_279(8156687) done Nov 04 22:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_298(8156706) WEXITSTATUS 0 Nov 04 22:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_298(8156706) done Nov 04 22:02:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_280(8156688) WEXITSTATUS 0 Nov 04 22:02:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_280(8156688) done Nov 04 22:02:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_303(8156741) WEXITSTATUS 0 Nov 04 22:02:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_303(8156741) done Nov 04 22:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_267(8156675) WEXITSTATUS 0 Nov 04 22:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_267(8156675) done Nov 04 22:02:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_271(8156679) WEXITSTATUS 0 Nov 04 22:02:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_271(8156679) done Nov 04 22:02:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_16(8158318) in day on r807u11n02 Nov 04 22:02:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_17(8158319) in day on r918u05n04 Nov 04 22:02:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_18(8158320) in day on r919u01n03 Nov 04 22:02:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_19(8158321) in day on r919u09n02 Nov 04 22:02:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r904u04n04 Nov 04 22:02:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r904u04n04 Nov 04 22:02:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_160(7572328) WEXITSTATUS 0 Nov 04 22:02:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_160(7572328) done Nov 04 22:02:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_85(8156438) WEXITSTATUS 0 Nov 04 22:02:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_85(8156438) done Nov 04 22:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157500 in scavenge on r816u09n12 Nov 04 22:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157501 in scavenge on r816u09n12 Nov 04 22:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157502 in scavenge on r816u09n12 Nov 04 22:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157503 in scavenge on r816u15n06 Nov 04 22:02:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_286(8156694) WEXITSTATUS 0 Nov 04 22:02:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_286(8156694) done Nov 04 22:02:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_362(8156800) WEXITSTATUS 0 Nov 04 22:02:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_362(8156800) done Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5026(8158322) in scavenge on r907u05n03 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5027(8158323) in scavenge on r915u35n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5028(8158324) in scavenge on r802u06n03 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5029(8158325) in scavenge on r802u23n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5030(8158326) in scavenge on r801u02n04 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5031(8158327) in scavenge on r801u06n01 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5032(8158328) in scavenge on r801u09n04 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5033(8158329) in scavenge on r806u06n01 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5034(8158330) in scavenge on r807u11n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5035(8158331) in scavenge on r807u11n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5036(8158332) in scavenge on r914u05n01 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5037(8158333) in scavenge on r914u05n01 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5038(8158334) in scavenge on r912u28n04 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5039(8158335) in scavenge on r912u36n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5040(8158336) in scavenge on r805u02n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5041(8158337) in scavenge on r805u09n01 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5042(8158338) in scavenge on r805u11n01 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5043(8158339) in scavenge on r918u01n03 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5044(8158340) in scavenge on r919u01n03 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5045(8158341) in scavenge on r919u05n04 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5046(8158342) in scavenge on r919u05n04 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5047(8158343) in scavenge on r919u05n04 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5048(8158344) in scavenge on r919u09n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5049(8158345) in scavenge on r919u09n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5050(8158346) in scavenge on r919u09n04 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5051(8158347) in scavenge on r919u09n04 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5052(8158348) in scavenge on r919u36n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5053(8158349) in scavenge on r919u36n02 Nov 04 22:02:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5054(8158350) in scavenge on r815u15n07 Nov 04 22:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_301(7572469) in scavenge on r906u25n01 Nov 04 22:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_79(8156432) WEXITSTATUS 0 Nov 04 22:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_79(8156432) done Nov 04 22:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_301(8156739) WEXITSTATUS 0 Nov 04 22:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_301(8156739) done Nov 04 22:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_228(8156636) WEXITSTATUS 0 Nov 04 22:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_228(8156636) done Nov 04 22:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_299(8156707) WEXITSTATUS 0 Nov 04 22:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_299(8156707) done Nov 04 22:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 04 22:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 04 22:03:01 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 04 22:03:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1810(8158351) in scavenge on r816u35n06 Nov 04 22:03:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1811(8158352) in scavenge on r816u35n06 Nov 04 22:03:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1812(8158353) in scavenge on r816u35n06 Nov 04 22:03:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_194(8156571) WEXITSTATUS 0 Nov 04 22:03:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_194(8156571) done Nov 04 22:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_483(8156948) WEXITSTATUS 0 Nov 04 22:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_483(8156948) done Nov 04 22:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_140(8156517) WEXITSTATUS 0 Nov 04 22:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_140(8156517) done Nov 04 22:03:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_452(8156917) WEXITSTATUS 0 Nov 04 22:03:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_452(8156917) done Nov 04 22:03:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_476(8156941) WEXITSTATUS 0 Nov 04 22:03:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_476(8156941) done Nov 04 22:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_473(8156938) WEXITSTATUS 0 Nov 04 22:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_473(8156938) done Nov 04 22:03:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_71(8156424) WEXITSTATUS 0 Nov 04 22:03:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_71(8156424) done Nov 04 22:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_145(8156522) WEXITSTATUS 0 Nov 04 22:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_145(8156522) done Nov 04 22:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_477(8156942) WEXITSTATUS 0 Nov 04 22:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_477(8156942) done Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:03:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:03:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5046(8158342) WEXITSTATUS 1 Nov 04 22:03:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5046(8158342) done Nov 04 22:03:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_89(8156442) WEXITSTATUS 0 Nov 04 22:03:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_89(8156442) done Nov 04 22:03:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_451(8156916) WEXITSTATUS 0 Nov 04 22:03:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_451(8156916) done Nov 04 22:03:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_467(8156932) WEXITSTATUS 0 Nov 04 22:03:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_467(8156932) done Nov 04 22:03:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_469(8156934) WEXITSTATUS 0 Nov 04 22:03:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_469(8156934) done Nov 04 22:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_260(8156668) WEXITSTATUS 0 Nov 04 22:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_260(8156668) done Nov 04 22:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_258(8156666) WEXITSTATUS 0 Nov 04 22:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_258(8156666) done Nov 04 22:03:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_578(8157063) WEXITSTATUS 0 Nov 04 22:03:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_578(8157063) done Nov 04 22:03:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_138(8156515) WEXITSTATUS 0 Nov 04 22:03:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_138(8156515) done Nov 04 22:03:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4288(8154054) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:03:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4677(8156339) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:03:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4288(8154054) Nov 04 22:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4558(8156015) WEXITSTATUS 1 Nov 04 22:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4558(8156015) done Nov 04 22:03:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4677(8156339) Nov 04 22:03:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_281(8156689) WEXITSTATUS 0 Nov 04 22:03:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_281(8156689) done Nov 04 22:03:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_282(8156690) WEXITSTATUS 0 Nov 04 22:03:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_282(8156690) done Nov 04 22:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_261(8156669) WEXITSTATUS 0 Nov 04 22:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_261(8156669) done Nov 04 22:03:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156013 WEXITSTATUS 0 Nov 04 22:03:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156013 done Nov 04 22:03:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_119(8156496) WEXITSTATUS 0 Nov 04 22:03:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_119(8156496) done Nov 04 22:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_446(8156911) WEXITSTATUS 0 Nov 04 22:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_446(8156911) done Nov 04 22:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_470(8156935) WEXITSTATUS 0 Nov 04 22:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_470(8156935) done Nov 04 22:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1852(8151178) WEXITSTATUS 0 Nov 04 22:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1852(8151178) done Nov 04 22:03:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_200(8156608) WEXITSTATUS 0 Nov 04 22:03:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_200(8156608) done Nov 04 22:03:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_233(8156641) WEXITSTATUS 0 Nov 04 22:03:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_233(8156641) done Nov 04 22:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4494(8155867) WEXITSTATUS 1 Nov 04 22:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4494(8155867) done Nov 04 22:03:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_82(8156435) WEXITSTATUS 0 Nov 04 22:03:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_82(8156435) done Nov 04 22:03:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_472(8156937) WEXITSTATUS 0 Nov 04 22:03:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_472(8156937) done Nov 04 22:03:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_272(8156680) WEXITSTATUS 0 Nov 04 22:03:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_272(8156680) done Nov 04 22:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_444(8156909) WEXITSTATUS 0 Nov 04 22:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_444(8156909) done Nov 04 22:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4991(8158264) WEXITSTATUS 1 Nov 04 22:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4991(8158264) done Nov 04 22:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_478(8156943) WEXITSTATUS 0 Nov 04 22:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_478(8156943) done Nov 04 22:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_137(8156514) WEXITSTATUS 0 Nov 04 22:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_137(8156514) done Nov 04 22:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_572(8157057) WEXITSTATUS 0 Nov 04 22:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_572(8157057) done Nov 04 22:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_582(8157067) WEXITSTATUS 0 Nov 04 22:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_582(8157067) done Nov 04 22:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_443(8156908) WEXITSTATUS 0 Nov 04 22:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_443(8156908) done Nov 04 22:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_479(8156944) WEXITSTATUS 0 Nov 04 22:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_479(8156944) done Nov 04 22:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_169(8156546) WEXITSTATUS 0 Nov 04 22:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_169(8156546) done Nov 04 22:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_139(8156516) WEXITSTATUS 0 Nov 04 22:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_139(8156516) done Nov 04 22:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_78(8156431) WEXITSTATUS 0 Nov 04 22:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_78(8156431) done Nov 04 22:04:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157504 NodeList=r816u03n12 #CPUs=12 Partition=scavenge Nov 04 22:04:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_448(8156913) WEXITSTATUS 0 Nov 04 22:04:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_448(8156913) done Nov 04 22:04:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_238(8156646) WEXITSTATUS 0 Nov 04 22:04:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_238(8156646) done Nov 04 22:04:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_142(8156519) WEXITSTATUS 0 Nov 04 22:04:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_142(8156519) done Nov 04 22:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_449(8156914) WEXITSTATUS 0 Nov 04 22:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_449(8156914) done Nov 04 22:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_242(8156650) WEXITSTATUS 0 Nov 04 22:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_242(8156650) done Nov 04 22:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_136(8156513) WEXITSTATUS 0 Nov 04 22:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_136(8156513) done Nov 04 22:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_306(8156744) WEXITSTATUS 0 Nov 04 22:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_306(8156744) done Nov 04 22:04:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5047(8158343) WEXITSTATUS 1 Nov 04 22:04:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5047(8158343) done Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_193(7572361) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_208(7572376) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_218(7572386) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_226(7572394) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_232(7572400) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_238(7572406) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_357(7572525) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_868(7573037) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1020(7573189) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_37(8114303) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_190(8114456) Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8124667 Nov 04 22:04:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8124668 Nov 04 22:04:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5000(8158284) WEXITSTATUS 1 Nov 04 22:04:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5000(8158284) done Nov 04 22:04:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_474(8156939) WEXITSTATUS 0 Nov 04 22:04:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_474(8156939) done Nov 04 22:04:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_216(8156624) WEXITSTATUS 0 Nov 04 22:04:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_216(8156624) done Nov 04 22:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_445(8156910) WEXITSTATUS 0 Nov 04 22:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_445(8156910) done Nov 04 22:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_2(8157550) WEXITSTATUS 0 Nov 04 22:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_2(8157550) done Nov 04 22:04:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_579(8157064) WEXITSTATUS 0 Nov 04 22:04:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_579(8157064) done Nov 04 22:04:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_352(8156790) WEXITSTATUS 0 Nov 04 22:04:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_352(8156790) done Nov 04 22:04:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_235(8156643) WEXITSTATUS 0 Nov 04 22:04:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_235(8156643) done Nov 04 22:04:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4685(8156347) WEXITSTATUS 1 Nov 04 22:04:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4685(8156347) done Nov 04 22:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_359(8156797) WEXITSTATUS 0 Nov 04 22:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_359(8156797) done Nov 04 22:04:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_357(8156795) WEXITSTATUS 0 Nov 04 22:04:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_357(8156795) done Nov 04 22:04:57 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8135100 Nov 04 22:04:58 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6676680_3(6676843) Nov 04 22:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_353(8156791) WEXITSTATUS 0 Nov 04 22:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_353(8156791) done Nov 04 22:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_264(8156672) WEXITSTATUS 0 Nov 04 22:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_264(8156672) done Nov 04 22:05:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_49(8156172) WEXITSTATUS 0 Nov 04 22:05:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_49(8156172) done Nov 04 22:05:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_263(8156671) WEXITSTATUS 0 Nov 04 22:05:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_263(8156671) done Nov 04 22:05:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_50(8156123) WEXITSTATUS 0 Nov 04 22:05:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_50(8156123) done Nov 04 22:05:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_581(8157066) WEXITSTATUS 0 Nov 04 22:05:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_581(8157066) done Nov 04 22:05:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_447(8156912) WEXITSTATUS 0 Nov 04 22:05:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_447(8156912) done Nov 04 22:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_365(8156803) WEXITSTATUS 0 Nov 04 22:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_365(8156803) done Nov 04 22:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_393(8156831) WEXITSTATUS 0 Nov 04 22:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_393(8156831) done Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:05:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_370(8156808) WEXITSTATUS 0 Nov 04 22:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_370(8156808) done Nov 04 22:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_366(8156804) WEXITSTATUS 0 Nov 04 22:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_366(8156804) done Nov 04 22:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_236(8156644) WEXITSTATUS 0 Nov 04 22:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_236(8156644) done Nov 04 22:05:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_240(8156648) WEXITSTATUS 0 Nov 04 22:05:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_240(8156648) done Nov 04 22:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2240(8152032) WEXITSTATUS 0 Nov 04 22:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2240(8152032) done Nov 04 22:05:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_577(8157062) WEXITSTATUS 0 Nov 04 22:05:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_577(8157062) done Nov 04 22:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_484(8156949) WEXITSTATUS 0 Nov 04 22:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_484(8156949) done Nov 04 22:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_395(8156833) WEXITSTATUS 0 Nov 04 22:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_395(8156833) done Nov 04 22:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_499(8156964) WEXITSTATUS 0 Nov 04 22:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_499(8156964) done Nov 04 22:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_392(8156830) WEXITSTATUS 0 Nov 04 22:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_392(8156830) done Nov 04 22:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158125_20(8158125) in day on r807u08n04 Nov 04 22:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_1(8158354) in day on r909u04n01 Nov 04 22:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_2(8158355) in day on r909u04n03 Nov 04 22:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_3(8158356) in day on r806u06n04 Nov 04 22:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_4(8158357) in day on r806u11n04 Nov 04 22:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_5(8158358) in day on r806u14n02 Nov 04 22:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_995(8115264) WEXITSTATUS 0 Nov 04 22:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_995(8115264) done Nov 04 22:05:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1813(8158359) in day on r914u01n01 Nov 04 22:05:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n04 Nov 04 22:05:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157505 in scavenge on r816u09n03 Nov 04 22:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_576(8157061) WEXITSTATUS 0 Nov 04 22:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_576(8157061) done Nov 04 22:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_356(8156794) WEXITSTATUS 0 Nov 04 22:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_356(8156794) done Nov 04 22:05:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_239(8156647) WEXITSTATUS 0 Nov 04 22:05:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_239(8156647) done Nov 04 22:05:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4837(8157834) WEXITSTATUS 1 Nov 04 22:05:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4837(8157834) done Nov 04 22:05:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_394(8156832) WEXITSTATUS 0 Nov 04 22:05:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_394(8156832) done Nov 04 22:05:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156029 WEXITSTATUS 0 Nov 04 22:05:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156029 done Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5055(8158360) in scavenge on r802u32n02 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5056(8158361) in scavenge on r912u01n01 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5057(8158362) in scavenge on r805u06n03 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5058(8158363) in scavenge on r904u07n04 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5059(8158364) in scavenge on r913u09n03 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5060(8158365) in scavenge on r913u09n03 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5061(8158366) in scavenge on r914u05n02 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5062(8158367) in scavenge on r919u05n04 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5063(8158368) in scavenge on r919u05n04 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5064(8158369) in scavenge on r909u04n03 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5065(8158370) in scavenge on r909u13n04 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5066(8158371) in scavenge on r805u38n01 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5067(8158372) in scavenge on r806u41n01 Nov 04 22:05:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5068(8158373) in scavenge on r806u41n01 Nov 04 22:05:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_78(7572245) in scavenge on r914u01n04 Nov 04 22:05:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_497(8156962) WEXITSTATUS 0 Nov 04 22:05:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_497(8156962) done Nov 04 22:05:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_583(8157068) WEXITSTATUS 0 Nov 04 22:05:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_583(8157068) done Nov 04 22:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_374(8156812) WEXITSTATUS 0 Nov 04 22:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_374(8156812) done Nov 04 22:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_372(8156810) WEXITSTATUS 0 Nov 04 22:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_372(8156810) done Nov 04 22:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_462(8156927) WEXITSTATUS 0 Nov 04 22:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_462(8156927) done Nov 04 22:05:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_460(8156925) WEXITSTATUS 0 Nov 04 22:05:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_460(8156925) done Nov 04 22:05:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_30(8148025) Nov 04 22:05:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_31(8148026) Nov 04 22:05:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_32(8148027) Nov 04 22:05:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_33(8148028) Nov 04 22:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_456(8156921) WEXITSTATUS 0 Nov 04 22:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_456(8156921) done Nov 04 22:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_224(8156632) WEXITSTATUS 0 Nov 04 22:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_224(8156632) done Nov 04 22:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8149445 WEXITSTATUS 0 Nov 04 22:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8149445 done Nov 04 22:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_588(8157073) WEXITSTATUS 0 Nov 04 22:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_588(8157073) done Nov 04 22:06:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1428(8135961) in scavenge on r806u41n01 Nov 04 22:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_592(8157077) WEXITSTATUS 0 Nov 04 22:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_592(8157077) done Nov 04 22:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5039(8158335) WEXITSTATUS 1 Nov 04 22:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5039(8158335) done Nov 04 22:06:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_440(8156905) WEXITSTATUS 0 Nov 04 22:06:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_440(8156905) done Nov 04 22:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_237(8156645) WEXITSTATUS 0 Nov 04 22:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_237(8156645) done Nov 04 22:06:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157506 NodeList=r816u29n07 #CPUs=12 Partition=scavenge Nov 04 22:06:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_311(8156749) WEXITSTATUS 0 Nov 04 22:06:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_311(8156749) done Nov 04 22:06:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_571(8157056) WEXITSTATUS 0 Nov 04 22:06:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_571(8157056) done Nov 04 22:06:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_345(8156783) WEXITSTATUS 0 Nov 04 22:06:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_345(8156783) done Nov 04 22:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_304(8156742) WEXITSTATUS 0 Nov 04 22:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_304(8156742) done Nov 04 22:06:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_305(8156743) WEXITSTATUS 0 Nov 04 22:06:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_305(8156743) done Nov 04 22:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 04 22:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 04 22:06:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 04 22:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4713(8156724) WEXITSTATUS 1 Nov 04 22:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4713(8156724) done Nov 04 22:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_376(8156814) WEXITSTATUS 0 Nov 04 22:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_376(8156814) done Nov 04 22:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_587(8157072) WEXITSTATUS 0 Nov 04 22:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_587(8157072) done Nov 04 22:06:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_234(8156642) WEXITSTATUS 0 Nov 04 22:06:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_234(8156642) done Nov 04 22:06:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_584(8157069) WEXITSTATUS 0 Nov 04 22:06:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_584(8157069) done Nov 04 22:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_241(8156649) WEXITSTATUS 0 Nov 04 22:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_241(8156649) done Nov 04 22:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_575(8157060) WEXITSTATUS 0 Nov 04 22:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_575(8157060) done Nov 04 22:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_498(8156963) WEXITSTATUS 0 Nov 04 22:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_498(8156963) done Nov 04 22:06:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_227(8156635) WEXITSTATUS 0 Nov 04 22:06:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_227(8156635) done Nov 04 22:06:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_580(8157065) WEXITSTATUS 0 Nov 04 22:06:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_580(8157065) done Nov 04 22:06:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_215(8156623) WEXITSTATUS 0 Nov 04 22:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_215(8156623) done Nov 04 22:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_369(8156807) WEXITSTATUS 0 Nov 04 22:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_369(8156807) done Nov 04 22:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_363(8156801) WEXITSTATUS 0 Nov 04 22:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_363(8156801) done Nov 04 22:06:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_461(8156926) WEXITSTATUS 0 Nov 04 22:06:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_461(8156926) done Nov 04 22:06:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_201(8156609) WEXITSTATUS 0 Nov 04 22:06:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_201(8156609) done Nov 04 22:06:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_367(8156805) WEXITSTATUS 0 Nov 04 22:06:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_367(8156805) done Nov 04 22:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_355(8156793) WEXITSTATUS 0 Nov 04 22:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_355(8156793) done Nov 04 22:06:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_589(8157074) WEXITSTATUS 0 Nov 04 22:06:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_589(8157074) done Nov 04 22:06:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_371(8156809) WEXITSTATUS 0 Nov 04 22:06:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_371(8156809) done Nov 04 22:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_346(8156784) WEXITSTATUS 0 Nov 04 22:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_346(8156784) done Nov 04 22:06:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2327(8152244) WEXITSTATUS 0 Nov 04 22:06:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2327(8152244) done Nov 04 22:06:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_47(8156400) WEXITSTATUS 0 Nov 04 22:06:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_47(8156400) done Nov 04 22:06:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_318(8156756) WEXITSTATUS 0 Nov 04 22:06:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_318(8156756) done Nov 04 22:06:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157914 WEXITSTATUS 0 Nov 04 22:06:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157914 done Nov 04 22:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_368(8156806) WEXITSTATUS 0 Nov 04 22:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_368(8156806) done Nov 04 22:06:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4512(8155901) WEXITSTATUS 1 Nov 04 22:06:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4512(8155901) done Nov 04 22:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_433(8156898) WEXITSTATUS 0 Nov 04 22:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_433(8156898) done Nov 04 22:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_436(8156901) WEXITSTATUS 0 Nov 04 22:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_436(8156901) done Nov 04 22:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4527(8155928) WEXITSTATUS 1 Nov 04 22:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4527(8155928) done Nov 04 22:06:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_439(8156904) WEXITSTATUS 0 Nov 04 22:06:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_439(8156904) done Nov 04 22:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_442(8156907) WEXITSTATUS 0 Nov 04 22:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_442(8156907) done Nov 04 22:06:59 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:07:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_496(8156961) WEXITSTATUS 0 Nov 04 22:07:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_496(8156961) done Nov 04 22:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_574(8157059) WEXITSTATUS 0 Nov 04 22:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_574(8157059) done Nov 04 22:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_212(8156620) WEXITSTATUS 0 Nov 04 22:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_212(8156620) done Nov 04 22:07:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156028 WEXITSTATUS 0 Nov 04 22:07:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156028 done Nov 04 22:07:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_593(8157078) WEXITSTATUS 0 Nov 04 22:07:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_593(8157078) done Nov 04 22:07:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_586(8157071) WEXITSTATUS 0 Nov 04 22:07:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_586(8157071) done Nov 04 22:07:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_396(8156834) WEXITSTATUS 0 Nov 04 22:07:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_396(8156834) done Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_225(8156633) WEXITSTATUS 0 Nov 04 22:07:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_225(8156633) done Nov 04 22:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_379(8156817) WEXITSTATUS 0 Nov 04 22:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_379(8156817) done Nov 04 22:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_493(8156958) WEXITSTATUS 0 Nov 04 22:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_493(8156958) done Nov 04 22:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_492(8156957) WEXITSTATUS 0 Nov 04 22:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_492(8156957) done Nov 04 22:07:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_454(8156919) WEXITSTATUS 0 Nov 04 22:07:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_454(8156919) done Nov 04 22:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157892 WEXITSTATUS 0 Nov 04 22:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157892 done Nov 04 22:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_207(8156615) WEXITSTATUS 0 Nov 04 22:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_207(8156615) done Nov 04 22:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_495(8156960) WEXITSTATUS 0 Nov 04 22:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_495(8156960) done Nov 04 22:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_377(8156815) WEXITSTATUS 0 Nov 04 22:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_377(8156815) done Nov 04 22:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_315(8156753) WEXITSTATUS 0 Nov 04 22:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_315(8156753) done Nov 04 22:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_491(8156956) WEXITSTATUS 0 Nov 04 22:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_491(8156956) done Nov 04 22:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_316(8156754) WEXITSTATUS 0 Nov 04 22:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_316(8156754) done Nov 04 22:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_210(8156618) WEXITSTATUS 0 Nov 04 22:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_210(8156618) done Nov 04 22:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_595(8157080) WEXITSTATUS 0 Nov 04 22:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_595(8157080) done Nov 04 22:07:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_375(8156813) WEXITSTATUS 0 Nov 04 22:07:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_375(8156813) done Nov 04 22:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_259(8156667) WEXITSTATUS 0 Nov 04 22:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_259(8156667) done Nov 04 22:07:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_387(8156825) WEXITSTATUS 0 Nov 04 22:07:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_387(8156825) done Nov 04 22:07:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_776(7571741) Nov 04 22:07:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1150(7573319) Nov 04 22:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_457(8156922) WEXITSTATUS 0 Nov 04 22:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_457(8156922) done Nov 04 22:07:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 04 22:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_354(8156792) WEXITSTATUS 0 Nov 04 22:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_354(8156792) done Nov 04 22:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_378(8156816) WEXITSTATUS 0 Nov 04 22:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_378(8156816) done Nov 04 22:07:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_590(8157075) WEXITSTATUS 0 Nov 04 22:07:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_590(8157075) done Nov 04 22:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_573(8157058) WEXITSTATUS 0 Nov 04 22:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_573(8157058) done Nov 04 22:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_390(8156828) WEXITSTATUS 0 Nov 04 22:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_390(8156828) done Nov 04 22:07:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_28(8156381) WEXITSTATUS 0 Nov 04 22:07:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_28(8156381) done Nov 04 22:07:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_585(8157070) WEXITSTATUS 0 Nov 04 22:07:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_585(8157070) done Nov 04 22:07:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_226(8156634) WEXITSTATUS 0 Nov 04 22:07:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_226(8156634) done Nov 04 22:07:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_213(8156621) WEXITSTATUS 0 Nov 04 22:07:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_213(8156621) done Nov 04 22:07:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_531(8157016) WEXITSTATUS 0 Nov 04 22:07:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_531(8157016) done Nov 04 22:07:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_223(8156631) WEXITSTATUS 0 Nov 04 22:07:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_223(8156631) done Nov 04 22:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_533(8157018) WEXITSTATUS 0 Nov 04 22:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_533(8157018) done Nov 04 22:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_532(8157017) WEXITSTATUS 0 Nov 04 22:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_532(8157017) done Nov 04 22:07:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_597(8157082) WEXITSTATUS 0 Nov 04 22:07:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_597(8157082) done Nov 04 22:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_214(8156622) WEXITSTATUS 0 Nov 04 22:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_214(8156622) done Nov 04 22:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_596(8157081) WEXITSTATUS 0 Nov 04 22:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_596(8157081) done Nov 04 22:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_320(8156758) WEXITSTATUS 0 Nov 04 22:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_320(8156758) done Nov 04 22:07:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_464(8156929) WEXITSTATUS 0 Nov 04 22:07:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_464(8156929) done Nov 04 22:07:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158374 InitPrio=33245 usec=3894 Nov 04 22:07:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_591(8157076) WEXITSTATUS 0 Nov 04 22:07:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_591(8157076) done Nov 04 22:07:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_594(8157079) WEXITSTATUS 0 Nov 04 22:07:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_594(8157079) done Nov 04 22:07:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_314(8156752) WEXITSTATUS 0 Nov 04 22:07:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_314(8156752) done Nov 04 22:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_203(8156611) WEXITSTATUS 0 Nov 04 22:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_203(8156611) done Nov 04 22:08:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_599(8157084) WEXITSTATUS 0 Nov 04 22:08:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_599(8157084) done Nov 04 22:08:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_373(8156811) WEXITSTATUS 0 Nov 04 22:08:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_373(8156811) done Nov 04 22:08:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_437(8156902) WEXITSTATUS 0 Nov 04 22:08:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_437(8156902) done Nov 04 22:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_620(8157126) WEXITSTATUS 0 Nov 04 22:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_620(8157126) done Nov 04 22:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_204(8156612) WEXITSTATUS 0 Nov 04 22:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_204(8156612) done Nov 04 22:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_403(8156868) WEXITSTATUS 0 Nov 04 22:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_403(8156868) done Nov 04 22:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_380(8156818) WEXITSTATUS 0 Nov 04 22:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_380(8156818) done Nov 04 22:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8137579 WEXITSTATUS 0 Nov 04 22:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8137579 done Nov 04 22:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_317(8156755) WEXITSTATUS 0 Nov 04 22:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_317(8156755) done Nov 04 22:08:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_337(8156775) WEXITSTATUS 0 Nov 04 22:08:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_337(8156775) done Nov 04 22:08:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_206(8156614) WEXITSTATUS 0 Nov 04 22:08:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_206(8156614) done Nov 04 22:08:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157507 NodeList=r816u23n12 #CPUs=12 Partition=scavenge Nov 04 22:08:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_598(8157083) WEXITSTATUS 0 Nov 04 22:08:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_598(8157083) done Nov 04 22:08:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_490(8156955) WEXITSTATUS 0 Nov 04 22:08:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_490(8156955) done Nov 04 22:08:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_534(8157019) WEXITSTATUS 0 Nov 04 22:08:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_534(8157019) done Nov 04 22:08:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_455(8156920) WEXITSTATUS 0 Nov 04 22:08:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_455(8156920) done Nov 04 22:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_360(8156798) WEXITSTATUS 0 Nov 04 22:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_360(8156798) done Nov 04 22:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_441(8156906) WEXITSTATUS 0 Nov 04 22:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_441(8156906) done Nov 04 22:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_166(8156543) WEXITSTATUS 0 Nov 04 22:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_166(8156543) done Nov 04 22:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_623(8157129) WEXITSTATUS 0 Nov 04 22:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_623(8157129) done Nov 04 22:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4975(8158237) WEXITSTATUS 1 Nov 04 22:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4975(8158237) done Nov 04 22:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_385(8156823) WEXITSTATUS 0 Nov 04 22:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_385(8156823) done Nov 04 22:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_205(8156613) WEXITSTATUS 0 Nov 04 22:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_205(8156613) done Nov 04 22:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_308(8156746) WEXITSTATUS 0 Nov 04 22:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_308(8156746) done Nov 04 22:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_6(8158375) in day on r913u13n04 Nov 04 22:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_7(8158376) in day on r918u36n03 Nov 04 22:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_8(8158377) in day on r806u09n01 Nov 04 22:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_9(8158378) in day on r806u11n03 Nov 04 22:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_10(8158379) in day on r816u15n02 Nov 04 22:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_11(8158380) in day on r816u15n02 Nov 04 22:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158374 in day on r904u07n01 Nov 04 22:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_219(8156627) WEXITSTATUS 0 Nov 04 22:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_219(8156627) done Nov 04 22:08:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6676680_3(6676843) in gpu on r808u17n01 Nov 04 22:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_402(8156867) WEXITSTATUS 0 Nov 04 22:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_402(8156867) done Nov 04 22:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_361(8156799) WEXITSTATUS 0 Nov 04 22:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_361(8156799) done Nov 04 22:08:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_172(8156549) WEXITSTATUS 0 Nov 04 22:08:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_172(8156549) done Nov 04 22:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_465(8156930) WEXITSTATUS 0 Nov 04 22:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_465(8156930) done Nov 04 22:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_222(8156630) WEXITSTATUS 0 Nov 04 22:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_222(8156630) done Nov 04 22:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_217(8156625) WEXITSTATUS 0 Nov 04 22:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_217(8156625) done Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_567(8157052) WEXITSTATUS 0 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_567(8157052) done Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_310(8156748) WEXITSTATUS 0 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_310(8156748) done Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5069(8158381) in scavenge on r802u23n04 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5070(8158382) in scavenge on r915u32n01 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5071(8158383) in scavenge on r906u28n01 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5072(8158384) in scavenge on r805u14n04 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5073(8158385) in scavenge on r806u14n04 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5074(8158386) in scavenge on r807u08n03 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5075(8158387) in scavenge on r909u04n03 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5076(8158388) in scavenge on r909u04n03 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5077(8158389) in scavenge on r914u01n01 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5078(8158390) in scavenge on r918u36n03 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5079(8158391) in scavenge on r918u36n03 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5080(8158392) in scavenge on r912u36n02 Nov 04 22:08:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5081(8158393) in scavenge on r913u01n04 Nov 04 22:08:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_389(8156827) WEXITSTATUS 0 Nov 04 22:08:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_389(8156827) done Nov 04 22:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_350(8156788) WEXITSTATUS 0 Nov 04 22:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_350(8156788) done Nov 04 22:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_319(8156757) WEXITSTATUS 0 Nov 04 22:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_319(8156757) done Nov 04 22:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_391(8156829) WEXITSTATUS 0 Nov 04 22:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_391(8156829) done Nov 04 22:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_358(8156796) WEXITSTATUS 0 Nov 04 22:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_358(8156796) done Nov 04 22:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_383(8156821) WEXITSTATUS 0 Nov 04 22:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_383(8156821) done Nov 04 22:08:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_34(8148237) Nov 04 22:08:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_35(8148238) Nov 04 22:08:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_36(8148239) Nov 04 22:08:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_37(8148240) Nov 04 22:08:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_38(8148241) Nov 04 22:08:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_39(8148242) Nov 04 22:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157908 WEXITSTATUS 0 Nov 04 22:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157908 done Nov 04 22:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_407(8156872) WEXITSTATUS 0 Nov 04 22:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_407(8156872) done Nov 04 22:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_625(8157131) WEXITSTATUS 0 Nov 04 22:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_625(8157131) done Nov 04 22:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_384(8156822) WEXITSTATUS 0 Nov 04 22:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_384(8156822) done Nov 04 22:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_434(8156899) WEXITSTATUS 0 Nov 04 22:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_434(8156899) done Nov 04 22:09:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_347(8156785) WEXITSTATUS 0 Nov 04 22:09:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_347(8156785) done Nov 04 22:09:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_307(8156745) WEXITSTATUS 0 Nov 04 22:09:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_307(8156745) done Nov 04 22:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_342(8156780) WEXITSTATUS 0 Nov 04 22:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_342(8156780) done Nov 04 22:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_343(8156781) WEXITSTATUS 0 Nov 04 22:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_343(8156781) done Nov 04 22:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_339(8156777) WEXITSTATUS 0 Nov 04 22:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_339(8156777) done Nov 04 22:09:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_341(8156779) WEXITSTATUS 0 Nov 04 22:09:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_341(8156779) done Nov 04 22:09:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_386(8156824) WEXITSTATUS 0 Nov 04 22:09:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_386(8156824) done Nov 04 22:09:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_626(8157132) WEXITSTATUS 0 Nov 04 22:09:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_626(8157132) done Nov 04 22:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_388(8156826) WEXITSTATUS 0 Nov 04 22:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_388(8156826) done Nov 04 22:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_537(8157022) WEXITSTATUS 0 Nov 04 22:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_537(8157022) done Nov 04 22:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_208(8156616) WEXITSTATUS 0 Nov 04 22:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_208(8156616) done Nov 04 22:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_344(8156782) WEXITSTATUS 0 Nov 04 22:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_344(8156782) done Nov 04 22:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_382(8156820) WEXITSTATUS 0 Nov 04 22:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_382(8156820) done Nov 04 22:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_381(8156819) WEXITSTATUS 0 Nov 04 22:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_381(8156819) done Nov 04 22:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_463(8156928) WEXITSTATUS 0 Nov 04 22:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_463(8156928) done Nov 04 22:09:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_209(8156617) WEXITSTATUS 0 Nov 04 22:09:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_209(8156617) done Nov 04 22:09:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_202(8156610) WEXITSTATUS 0 Nov 04 22:09:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_202(8156610) done Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:09:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:09:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157927 WEXITSTATUS 0 Nov 04 22:09:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157927 done Nov 04 22:09:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_405(8156870) WEXITSTATUS 0 Nov 04 22:09:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_405(8156870) done Nov 04 22:09:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_312(8156750) WEXITSTATUS 0 Nov 04 22:09:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_312(8156750) done Nov 04 22:09:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3477(8157776) WEXITSTATUS 0 Nov 04 22:09:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3477(8157776) done Nov 04 22:09:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2099(8151775) WEXITSTATUS 0 Nov 04 22:09:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2099(8151775) done Nov 04 22:09:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_489(8156954) WEXITSTATUS 0 Nov 04 22:09:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_489(8156954) done Nov 04 22:09:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4707(8156590) WEXITSTATUS 1 Nov 04 22:09:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4707(8156590) done Nov 04 22:09:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_313(8156751) WEXITSTATUS 0 Nov 04 22:09:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_313(8156751) done Nov 04 22:09:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1824(8124704) Nov 04 22:09:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5036(8158332) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:09:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5037(8158333) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:09:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5036(8158332) Nov 04 22:09:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5037(8158333) Nov 04 22:09:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_619(8157125) WEXITSTATUS 0 Nov 04 22:09:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_619(8157125) done Nov 04 22:09:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3529(8158270) WEXITSTATUS 0 Nov 04 22:09:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3529(8158270) done Nov 04 22:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_535(8157020) WEXITSTATUS 0 Nov 04 22:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_535(8157020) done Nov 04 22:09:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_610(8157116) WEXITSTATUS 0 Nov 04 22:09:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_610(8157116) done Nov 04 22:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5057(8158362) WEXITSTATUS 1 Nov 04 22:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5057(8158362) done Nov 04 22:09:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_569(8157054) WEXITSTATUS 0 Nov 04 22:09:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_569(8157054) done Nov 04 22:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_606(8157112) WEXITSTATUS 0 Nov 04 22:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_606(8157112) done Nov 04 22:09:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4693(8156465) WEXITSTATUS 1 Nov 04 22:09:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4693(8156465) done Nov 04 22:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_309(8156747) WEXITSTATUS 0 Nov 04 22:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_309(8156747) done Nov 04 22:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_557(8157042) WEXITSTATUS 0 Nov 04 22:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_557(8157042) done Nov 04 22:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_627(8157133) WEXITSTATUS 0 Nov 04 22:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_627(8157133) done Nov 04 22:09:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_306(7572474) Nov 04 22:10:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_536(8157021) WEXITSTATUS 0 Nov 04 22:10:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_536(8157021) done Nov 04 22:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_458(8156923) WEXITSTATUS 0 Nov 04 22:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_458(8156923) done Nov 04 22:10:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4998(8158277) WEXITSTATUS 1 Nov 04 22:10:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4998(8158277) done Nov 04 22:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_570(8157055) WEXITSTATUS 0 Nov 04 22:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_570(8157055) done Nov 04 22:10:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_609(8157115) WEXITSTATUS 0 Nov 04 22:10:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_609(8157115) done Nov 04 22:10:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_349(8156787) WEXITSTATUS 0 Nov 04 22:10:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_349(8156787) done Nov 04 22:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_30(8156153) WEXITSTATUS 0 Nov 04 22:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_30(8156153) done Nov 04 22:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_616(8157122) WEXITSTATUS 0 Nov 04 22:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_616(8157122) done Nov 04 22:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_611(8157117) WEXITSTATUS 0 Nov 04 22:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_611(8157117) done Nov 04 22:10:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1822(8151148) WEXITSTATUS 0 Nov 04 22:10:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1822(8151148) done Nov 04 22:10:17 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:10:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:10:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:10:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_348(8156786) WEXITSTATUS 0 Nov 04 22:10:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_348(8156786) done Nov 04 22:10:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_338(8156776) WEXITSTATUS 0 Nov 04 22:10:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_338(8156776) done Nov 04 22:10:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4556(8155965) WEXITSTATUS 1 Nov 04 22:10:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4556(8155965) done Nov 04 22:10:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2233(8152025) WEXITSTATUS 0 Nov 04 22:10:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2233(8152025) done Nov 04 22:10:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_211(8156619) WEXITSTATUS 0 Nov 04 22:10:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_211(8156619) done Nov 04 22:10:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_404(8156869) WEXITSTATUS 0 Nov 04 22:10:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_404(8156869) done Nov 04 22:10:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157920 WEXITSTATUS 0 Nov 04 22:10:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157920 done Nov 04 22:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_525(8157010) WEXITSTATUS 0 Nov 04 22:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_525(8157010) done Nov 04 22:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_526(8157011) WEXITSTATUS 0 Nov 04 22:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_526(8157011) done Nov 04 22:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_47(8156170) WEXITSTATUS 0 Nov 04 22:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_47(8156170) done Nov 04 22:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_530(8157015) WEXITSTATUS 0 Nov 04 22:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_530(8157015) done Nov 04 22:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_618(8157124) WEXITSTATUS 0 Nov 04 22:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_618(8157124) done Nov 04 22:10:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Unspecified error Nov 04 22:10:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_336(8156774) WEXITSTATUS 0 Nov 04 22:10:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_336(8156774) done Nov 04 22:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_340(8156778) WEXITSTATUS 0 Nov 04 22:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_340(8156778) done Nov 04 22:10:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Unspecified error Nov 04 22:10:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158394 InitPrio=47154 usec=6706 Nov 04 22:10:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_568(8157053) WEXITSTATUS 0 Nov 04 22:10:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_568(8157053) done Nov 04 22:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_401(8156866) WEXITSTATUS 0 Nov 04 22:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_401(8156866) done Nov 04 22:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_524(8157009) WEXITSTATUS 0 Nov 04 22:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_524(8157009) done Nov 04 22:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_603(8157109) WEXITSTATUS 0 Nov 04 22:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_603(8157109) done Nov 04 22:10:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_406(8156871) WEXITSTATUS 0 Nov 04 22:10:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_406(8156871) done Nov 04 22:11:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_218(8156626) WEXITSTATUS 0 Nov 04 22:11:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_218(8156626) done Nov 04 22:11:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_529(8157014) WEXITSTATUS 0 Nov 04 22:11:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_529(8157014) done Nov 04 22:11:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_220(8156628) WEXITSTATUS 0 Nov 04 22:11:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_220(8156628) done Nov 04 22:11:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_330(8156768) WEXITSTATUS 0 Nov 04 22:11:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_330(8156768) done Nov 04 22:11:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_528(8157013) WEXITSTATUS 0 Nov 04 22:11:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_528(8157013) done Nov 04 22:11:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4908(8158058) WEXITSTATUS 1 Nov 04 22:11:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4908(8158058) done Nov 04 22:11:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158394 NodeList=r909u07n02 #CPUs=13 Partition=day Nov 04 22:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_438(8156903) WEXITSTATUS 0 Nov 04 22:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_438(8156903) done Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_538(8157023) WEXITSTATUS 0 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_538(8157023) done Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_435(8156900) WEXITSTATUS 0 Nov 04 22:11:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_435(8156900) done Nov 04 22:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4312(8154195) WEXITSTATUS 1 Nov 04 22:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4312(8154195) done Nov 04 22:11:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_400(8156865) WEXITSTATUS 0 Nov 04 22:11:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_400(8156865) done Nov 04 22:11:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5054(8158350) WEXITSTATUS 1 Nov 04 22:11:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5054(8158350) done Nov 04 22:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157904 WEXITSTATUS 0 Nov 04 22:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157904 done Nov 04 22:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_622(8157128) WEXITSTATUS 0 Nov 04 22:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_622(8157128) done Nov 04 22:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_556(8157041) WEXITSTATUS 0 Nov 04 22:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_556(8157041) done Nov 04 22:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_554(8157039) WEXITSTATUS 0 Nov 04 22:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_554(8157039) done Nov 04 22:11:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_621(8157127) WEXITSTATUS 0 Nov 04 22:11:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_621(8157127) done Nov 04 22:11:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_333(8156771) WEXITSTATUS 0 Nov 04 22:11:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_333(8156771) done Nov 04 22:11:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_624(8157130) WEXITSTATUS 0 Nov 04 22:11:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_624(8157130) done Nov 04 22:11:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_605(8157111) WEXITSTATUS 0 Nov 04 22:11:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_605(8157111) done Nov 04 22:11:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_541(8157026) WEXITSTATUS 0 Nov 04 22:11:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_541(8157026) done Nov 04 22:11:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_334(8156772) WEXITSTATUS 0 Nov 04 22:11:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_334(8156772) done Nov 04 22:11:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4042(8147095) WEXITSTATUS 1 Nov 04 22:11:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4042(8147095) done Nov 04 22:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_539(8157024) WEXITSTATUS 0 Nov 04 22:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_539(8157024) done Nov 04 22:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_552(8157037) WEXITSTATUS 0 Nov 04 22:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_552(8157037) done Nov 04 22:11:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5031(8158327) WEXITSTATUS 1 Nov 04 22:11:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5031(8158327) done Nov 04 22:11:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158395 InitPrio=36224 usec=6082 Nov 04 22:11:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158395 in devel on r806u23n03 Nov 04 22:11:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4860(8157959) WEXITSTATUS 1 Nov 04 22:11:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4860(8157959) done Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_12(8158396) in day on r806u11n01 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_13(8158397) in day on r914u05n01 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_14(8158398) in day on r807u05n02 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_15(8158399) in day on r807u05n03 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_16(8158400) in day on r909u01n02 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_17(8158401) in day on r909u10n02 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_18(8158402) in day on r909u13n03 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_19(8158403) in day on r909u29n02 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3531(8158404) in gpu on r808u11n01 Nov 04 22:11:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3532(8158405) in gpu on r903u34n01 Nov 04 22:11:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n04 Nov 04 22:11:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_801(7571766) WEXITSTATUS 0 Nov 04 22:11:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_801(7571766) done Nov 04 22:11:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1861(8151187) WEXITSTATUS 0 Nov 04 22:11:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1861(8151187) done Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5082(8158406) in scavenge on r802u02n01 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5083(8158407) in scavenge on r917u10n01 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5084(8158408) in scavenge on r801u06n01 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5085(8158409) in scavenge on r801u14n02 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5086(8158410) in scavenge on r805u06n03 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5087(8158411) in scavenge on r913u24n04 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5088(8158412) in scavenge on r914u01n01 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5089(8158413) in scavenge on r918u36n04 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5090(8158414) in scavenge on r909u01n02 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5091(8158415) in scavenge on r909u10n02 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5092(8158416) in scavenge on r909u10n02 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5093(8158417) in scavenge on r909u10n02 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5094(8158418) in scavenge on r909u29n02 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5095(8158419) in scavenge on r909u29n02 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5096(8158420) in scavenge on r815u15n07 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5097(8158421) in scavenge on r815u23n02 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5098(8158422) in scavenge on r806u31n01 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3520(8158225) WEXITSTATUS 0 Nov 04 22:11:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3520(8158225) done Nov 04 22:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_324(8156762) WEXITSTATUS 0 Nov 04 22:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_324(8156762) done Nov 04 22:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_564(8157049) WEXITSTATUS 0 Nov 04 22:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_564(8157049) done Nov 04 22:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_607(8157113) WEXITSTATUS 0 Nov 04 22:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_607(8157113) done Nov 04 22:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_424(8156889) WEXITSTATUS 0 Nov 04 22:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_424(8156889) done Nov 04 22:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_613(8157119) WEXITSTATUS 0 Nov 04 22:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_613(8157119) done Nov 04 22:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_48(8156171) WEXITSTATUS 0 Nov 04 22:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_48(8156171) done Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_604(8157110) WEXITSTATUS 0 Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_604(8157110) done Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_329(8156767) WEXITSTATUS 0 Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_329(8156767) done Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_614(8157120) WEXITSTATUS 0 Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_614(8157120) done Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2266727316]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 04 22:11:49 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2266727316]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_617(8157123) WEXITSTATUS 0 Nov 04 22:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_617(8157123) done Nov 04 22:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5021(8158312) WEXITSTATUS 1 Nov 04 22:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5021(8158312) done Nov 04 22:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_331(8156769) WEXITSTATUS 0 Nov 04 22:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_331(8156769) done Nov 04 22:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_555(8157040) WEXITSTATUS 0 Nov 04 22:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_555(8157040) done Nov 04 22:11:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_321(8156759) WEXITSTATUS 0 Nov 04 22:11:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_321(8156759) done Nov 04 22:11:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_553(8157038) WEXITSTATUS 0 Nov 04 22:11:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_553(8157038) done Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_40(8148457) Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_41(8148458) Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_42(8148459) Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_43(8148460) Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_44(8148461) Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_45(8148462) Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_46(8148463) Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4971(8158233) WEXITSTATUS 1 Nov 04 22:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4971(8158233) done Nov 04 22:12:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_103(8156480) WEXITSTATUS 0 Nov 04 22:12:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_103(8156480) done Nov 04 22:12:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_431(8156896) WEXITSTATUS 0 Nov 04 22:12:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_431(8156896) done Nov 04 22:12:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157883 WEXITSTATUS 0 Nov 04 22:12:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157883 done Nov 04 22:12:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3532(8158405) WEXITSTATUS 0 Nov 04 22:12:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3532(8158405) done Nov 04 22:12:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_428(8156893) WEXITSTATUS 0 Nov 04 22:12:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_428(8156893) done Nov 04 22:12:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_323(8156761) WEXITSTATUS 0 Nov 04 22:12:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_323(8156761) done Nov 04 22:12:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2387(8126415) WEXITSTATUS 1 Nov 04 22:12:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2387(8126415) done Nov 04 22:12:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_328(8156766) WEXITSTATUS 0 Nov 04 22:12:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_328(8156766) done Nov 04 22:12:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_565(8157050) WEXITSTATUS 0 Nov 04 22:12:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_565(8157050) done Nov 04 22:12:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4748(8157098) WEXITSTATUS 1 Nov 04 22:12:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4748(8157098) done Nov 04 22:12:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_426(8156891) WEXITSTATUS 0 Nov 04 22:12:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_426(8156891) done Nov 04 22:12:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_75(8156428) WEXITSTATUS 0 Nov 04 22:12:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_75(8156428) done Nov 04 22:12:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7568129_1031(7569160) Nov 04 22:12:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_348(7572516) Nov 04 22:12:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1566(8150545) WEXITSTATUS 0 Nov 04 22:12:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1566(8150545) done Nov 04 22:12:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_615(8157121) WEXITSTATUS 0 Nov 04 22:12:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_615(8157121) done Nov 04 22:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_512(8156997) WEXITSTATUS 0 Nov 04 22:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_512(8156997) done Nov 04 22:12:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_432(8156897) WEXITSTATUS 0 Nov 04 22:12:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_432(8156897) done Nov 04 22:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_630(8157136) WEXITSTATUS 0 Nov 04 22:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_630(8157136) done Nov 04 22:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4245(8152912) WEXITSTATUS 1 Nov 04 22:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4245(8152912) done Nov 04 22:12:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_429(8156894) WEXITSTATUS 0 Nov 04 22:12:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_429(8156894) done Nov 04 22:12:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_427(8156892) WEXITSTATUS 0 Nov 04 22:12:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_427(8156892) done Nov 04 22:12:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157900 WEXITSTATUS 0 Nov 04 22:12:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157900 done Nov 04 22:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3966(8146009) WEXITSTATUS 1 Nov 04 22:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3966(8146009) done Nov 04 22:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_19(8156142) WEXITSTATUS 0 Nov 04 22:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_19(8156142) done Nov 04 22:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_332(8156770) WEXITSTATUS 0 Nov 04 22:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_332(8156770) done Nov 04 22:12:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158423 InitPrio=29946 usec=6847 Nov 04 22:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_13(8156136) WEXITSTATUS 0 Nov 04 22:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_13(8156136) done Nov 04 22:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_563(8157048) WEXITSTATUS 0 Nov 04 22:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_563(8157048) done Nov 04 22:12:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_146(8156523) WEXITSTATUS 0 Nov 04 22:12:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_146(8156523) done Nov 04 22:12:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_425(8156890) WEXITSTATUS 0 Nov 04 22:12:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_425(8156890) done Nov 04 22:12:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8135146 Nov 04 22:12:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157891 WEXITSTATUS 0 Nov 04 22:12:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157891 done Nov 04 22:12:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4934(8158121) WEXITSTATUS 1 Nov 04 22:12:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4934(8158121) done Nov 04 22:13:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_335(8156773) WEXITSTATUS 0 Nov 04 22:13:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_335(8156773) done Nov 04 22:13:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_325(8156763) WEXITSTATUS 0 Nov 04 22:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_325(8156763) done Nov 04 22:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157910 WEXITSTATUS 0 Nov 04 22:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157910 done Nov 04 22:13:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_540(8157025) WEXITSTATUS 0 Nov 04 22:13:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_540(8157025) done Nov 04 22:13:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_39(8156162) WEXITSTATUS 0 Nov 04 22:13:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_39(8156162) done Nov 04 22:13:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_527(8157012) WEXITSTATUS 0 Nov 04 22:13:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_527(8157012) done Nov 04 22:13:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_628(8157134) WEXITSTATUS 0 Nov 04 22:13:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_628(8157134) done Nov 04 22:13:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157508 NodeList=r816u29n02 #CPUs=12 Partition=scavenge Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_327(8156765) WEXITSTATUS 0 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_327(8156765) done Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_551(8157036) WEXITSTATUS 0 Nov 04 22:13:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_551(8157036) done Nov 04 22:13:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_12(8156135) WEXITSTATUS 0 Nov 04 22:13:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_12(8156135) done Nov 04 22:13:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157898 WEXITSTATUS 0 Nov 04 22:13:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157898 done Nov 04 22:13:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_608(8157114) WEXITSTATUS 0 Nov 04 22:13:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_608(8157114) done Nov 04 22:13:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_351(8156789) WEXITSTATUS 0 Nov 04 22:13:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_351(8156789) done Nov 04 22:13:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156030 WEXITSTATUS 0 Nov 04 22:13:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156030 done Nov 04 22:13:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_408(8156873) WEXITSTATUS 0 Nov 04 22:13:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_408(8156873) done Nov 04 22:13:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_26(8156149) WEXITSTATUS 0 Nov 04 22:13:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_26(8156149) done Nov 04 22:13:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_640(8157146) WEXITSTATUS 0 Nov 04 22:13:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_640(8157146) done Nov 04 22:13:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_629(8157135) WEXITSTATUS 0 Nov 04 22:13:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_629(8157135) done Nov 04 22:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4757(8157218) WEXITSTATUS 1 Nov 04 22:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4757(8157218) done Nov 04 22:13:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_430(8156895) WEXITSTATUS 0 Nov 04 22:13:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_430(8156895) done Nov 04 22:13:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2222(8152014) WEXITSTATUS 0 Nov 04 22:13:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2222(8152014) done Nov 04 22:13:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_612(8157118) WEXITSTATUS 0 Nov 04 22:13:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_612(8157118) done Nov 04 22:13:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157919 WEXITSTATUS 0 Nov 04 22:13:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157919 done Nov 04 22:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1823(8151149) WEXITSTATUS 0 Nov 04 22:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1823(8151149) done Nov 04 22:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_322(8156760) WEXITSTATUS 0 Nov 04 22:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_322(8156760) done Nov 04 22:13:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_631(8157137) WEXITSTATUS 0 Nov 04 22:13:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_631(8157137) done Nov 04 22:13:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157902 WEXITSTATUS 0 Nov 04 22:13:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157902 done Nov 04 22:13:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_600(8157106) WEXITSTATUS 0 Nov 04 22:13:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_600(8157106) done Nov 04 22:13:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_559(8157044) WEXITSTATUS 0 Nov 04 22:13:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_559(8157044) done Nov 04 22:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157890 WEXITSTATUS 0 Nov 04 22:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157890 done Nov 04 22:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5006(8158290) WEXITSTATUS 1 Nov 04 22:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5006(8158290) done Nov 04 22:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3810(8144424) WEXITSTATUS 1 Nov 04 22:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3810(8144424) done Nov 04 22:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_41(8156164) WEXITSTATUS 0 Nov 04 22:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_41(8156164) done Nov 04 22:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_1(8156124) WEXITSTATUS 0 Nov 04 22:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_1(8156124) done Nov 04 22:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5052(8158348) WEXITSTATUS 1 Nov 04 22:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5052(8158348) done Nov 04 22:14:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_413(8156878) WEXITSTATUS 0 Nov 04 22:14:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_413(8156878) done Nov 04 22:14:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_633(8157139) WEXITSTATUS 0 Nov 04 22:14:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_633(8157139) done Nov 04 22:14:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_660(8157166) WEXITSTATUS 0 Nov 04 22:14:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_660(8157166) done Nov 04 22:14:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_511(8156996) WEXITSTATUS 0 Nov 04 22:14:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_511(8156996) done Nov 04 22:14:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_522(8157007) WEXITSTATUS 0 Nov 04 22:14:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_522(8157007) done Nov 04 22:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_508(8156993) WEXITSTATUS 0 Nov 04 22:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_508(8156993) done Nov 04 22:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_326(8156764) WEXITSTATUS 0 Nov 04 22:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_326(8156764) done Nov 04 22:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5098(8158422) WEXITSTATUS 1 Nov 04 22:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5098(8158422) done Nov 04 22:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_42(8156165) WEXITSTATUS 0 Nov 04 22:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_42(8156165) done Nov 04 22:14:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157932 WEXITSTATUS 0 Nov 04 22:14:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157932 done Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158160_20(8158160) in day on r805u11n04 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158423 in day on r914u05n04 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3533(8158424) in gpu on r808u17n01 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3534(8158425) in gpu on r903u34n01 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1814(8158426) in day on r913u05n04 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1815(8158427) in day on r914u05n04 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1816(8158428) in day on r909u01n01 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1817(8158429) in day on r909u01n03 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1818(8158430) in day on r909u04n01 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1819(8158431) in day on r909u29n04 Nov 04 22:14:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157509 in scavenge on r815u23n06 Nov 04 22:14:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_504(8156989) WEXITSTATUS 0 Nov 04 22:14:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_504(8156989) done Nov 04 22:14:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_78(7572245) WEXITSTATUS 0 Nov 04 22:14:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_78(7572245) done Nov 04 22:14:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_998(7573167) WEXITSTATUS 0 Nov 04 22:14:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_998(7573167) done Nov 04 22:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157889 WEXITSTATUS 0 Nov 04 22:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157889 done Nov 04 22:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_658(8157164) WEXITSTATUS 0 Nov 04 22:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_658(8157164) done Nov 04 22:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_423(8156888) WEXITSTATUS 0 Nov 04 22:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_423(8156888) done Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5099(8158432) in scavenge on r907u01n04 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5100(8158433) in scavenge on r908u35n02 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5101(8158434) in scavenge on r802u09n03 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5102(8158435) in scavenge on r915u32n01 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5103(8158436) in scavenge on r805u04n02 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5104(8158437) in scavenge on r904u07n03 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5105(8158438) in scavenge on r913u05n04 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5106(8158439) in scavenge on r914u05n01 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5107(8158440) in scavenge on r909u01n03 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5108(8158441) in scavenge on r909u01n03 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5109(8158442) in scavenge on r909u01n03 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5110(8158443) in scavenge on r909u01n03 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5111(8158444) in scavenge on r909u04n01 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5112(8158445) in scavenge on r909u04n01 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5113(8158446) in scavenge on r909u04n01 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5114(8158447) in scavenge on r919u09n01 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5115(8158448) in scavenge on r919u32n03 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5116(8158449) in scavenge on r919u36n02 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5117(8158450) in scavenge on r815u15n03 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5118(8158451) in scavenge on r815u35n04 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5119(8158452) in scavenge on r806u31n01 Nov 04 22:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5120(8158453) in scavenge on r908u24n01 Nov 04 22:14:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4847(8157855) WEXITSTATUS 1 Nov 04 22:14:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4847(8157855) done Nov 04 22:14:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1832(8124746) Nov 04 22:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_39(8135738) WEXITSTATUS 0 Nov 04 22:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_39(8135738) done Nov 04 22:14:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_510(8156995) WEXITSTATUS 0 Nov 04 22:14:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_510(8156995) done Nov 04 22:14:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 04 22:14:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_870(8127984) in scavenge on r912u09n01 Nov 04 22:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_635(8157141) WEXITSTATUS 0 Nov 04 22:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_635(8157141) done Nov 04 22:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_519(8157004) WEXITSTATUS 0 Nov 04 22:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_519(8157004) done Nov 04 22:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157897 WEXITSTATUS 0 Nov 04 22:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157897 done Nov 04 22:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3534(8158425) WEXITSTATUS 0 Nov 04 22:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3534(8158425) done Nov 04 22:14:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_514(8156999) WEXITSTATUS 0 Nov 04 22:14:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_514(8156999) done Nov 04 22:14:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_657(8157163) WEXITSTATUS 0 Nov 04 22:14:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_657(8157163) done Nov 04 22:14:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_639(8157145) WEXITSTATUS 0 Nov 04 22:14:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_639(8157145) done Nov 04 22:14:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2305(8152222) WEXITSTATUS 0 Nov 04 22:14:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2305(8152222) done Nov 04 22:14:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_16(8156139) WEXITSTATUS 0 Nov 04 22:14:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_16(8156139) done Nov 04 22:14:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157929 WEXITSTATUS 0 Nov 04 22:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157929 done Nov 04 22:14:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_412(8156877) WEXITSTATUS 0 Nov 04 22:14:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_412(8156877) done Nov 04 22:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_417(8156882) WEXITSTATUS 0 Nov 04 22:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_417(8156882) done Nov 04 22:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5067(8158372) WEXITSTATUS 1 Nov 04 22:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5067(8158372) done Nov 04 22:14:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4789(8157664) WEXITSTATUS 1 Nov 04 22:14:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4789(8157664) done Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_293(7572461) Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_358(7572526) Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_359(7572527) Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8149757_33(8155773) had to be killed Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5023(8158314) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5024(8158315) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1767(8150978) WEXITSTATUS 0 Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1767(8150978) done Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157921 WEXITSTATUS 0 Nov 04 22:15:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157921 done Nov 04 22:15:01 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5024(8158315) Nov 04 22:15:02 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5023(8158314) Nov 04 22:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_520(8157005) WEXITSTATUS 0 Nov 04 22:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_520(8157005) done Nov 04 22:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_422(8156887) WEXITSTATUS 0 Nov 04 22:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_422(8156887) done Nov 04 22:15:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3777(8143574) WEXITSTATUS 1 Nov 04 22:15:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3777(8143574) done Nov 04 22:15:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157926 WEXITSTATUS 0 Nov 04 22:15:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157926 done Nov 04 22:15:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_560(8157045) WEXITSTATUS 0 Nov 04 22:15:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_560(8157045) done Nov 04 22:15:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_561(8157046) WEXITSTATUS 0 Nov 04 22:15:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_561(8157046) done Nov 04 22:15:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_421(8156886) WEXITSTATUS 0 Nov 04 22:15:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_421(8156886) done Nov 04 22:15:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3452(8157209) WEXITSTATUS 0 Nov 04 22:15:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3452(8157209) done Nov 04 22:15:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_562(8157047) WEXITSTATUS 0 Nov 04 22:15:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_562(8157047) done Nov 04 22:15:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_411(8156876) WEXITSTATUS 0 Nov 04 22:15:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_411(8156876) done Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_415(8156880) WEXITSTATUS 0 Nov 04 22:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_415(8156880) done Nov 04 22:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_414(8156879) WEXITSTATUS 0 Nov 04 22:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_414(8156879) done Nov 04 22:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157918 WEXITSTATUS 0 Nov 04 22:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157918 done Nov 04 22:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_416(8156881) WEXITSTATUS 0 Nov 04 22:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_416(8156881) done Nov 04 22:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_418(8156883) WEXITSTATUS 0 Nov 04 22:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_418(8156883) done Nov 04 22:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_896(8157439) WEXITSTATUS 0 Nov 04 22:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_896(8157439) done Nov 04 22:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_637(8157143) WEXITSTATUS 0 Nov 04 22:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_637(8157143) done Nov 04 22:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_634(8157140) WEXITSTATUS 0 Nov 04 22:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_634(8157140) done Nov 04 22:15:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8158454 NodeList=(null) usec=28444 Nov 04 22:15:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_509(8156994) WEXITSTATUS 0 Nov 04 22:15:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_509(8156994) done Nov 04 22:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_638(8157144) WEXITSTATUS 0 Nov 04 22:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_638(8157144) done Nov 04 22:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_505(8156990) WEXITSTATUS 0 Nov 04 22:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_505(8156990) done Nov 04 22:15:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158454 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 22:15:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_419(8156884) WEXITSTATUS 0 Nov 04 22:15:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_419(8156884) done Nov 04 22:15:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_893(8157436) WEXITSTATUS 0 Nov 04 22:15:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_893(8157436) done Nov 04 22:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_558(8157043) WEXITSTATUS 0 Nov 04 22:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_558(8157043) done Nov 04 22:15:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_506(8156991) WEXITSTATUS 0 Nov 04 22:15:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_506(8156991) done Nov 04 22:15:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_681(8157187) WEXITSTATUS 0 Nov 04 22:15:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_681(8157187) done Nov 04 22:15:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_9(8156132) WEXITSTATUS 0 Nov 04 22:15:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_9(8156132) done Nov 04 22:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2013(8151573) WEXITSTATUS 0 Nov 04 22:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2013(8151573) done Nov 04 22:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_507(8156992) WEXITSTATUS 0 Nov 04 22:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_507(8156992) done Nov 04 22:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1860(8151186) WEXITSTATUS 0 Nov 04 22:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1860(8151186) done Nov 04 22:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_679(8157185) WEXITSTATUS 0 Nov 04 22:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_679(8157185) done Nov 04 22:15:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_410(8156875) WEXITSTATUS 0 Nov 04 22:15:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_410(8156875) done Nov 04 22:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_683(8157189) WEXITSTATUS 0 Nov 04 22:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_683(8157189) done Nov 04 22:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_653(8157159) WEXITSTATUS 0 Nov 04 22:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_653(8157159) done Nov 04 22:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2006(8151566) WEXITSTATUS 0 Nov 04 22:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2006(8151566) done Nov 04 22:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_892(8157435) WEXITSTATUS 0 Nov 04 22:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_892(8157435) done Nov 04 22:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157912 WEXITSTATUS 0 Nov 04 22:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157912 done Nov 04 22:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_891(8157434) WEXITSTATUS 0 Nov 04 22:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_891(8157434) done Nov 04 22:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3530(8158282) WEXITSTATUS 0 Nov 04 22:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3530(8158282) done Nov 04 22:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_655(8157161) WEXITSTATUS 0 Nov 04 22:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_655(8157161) done Nov 04 22:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_36(8156159) WEXITSTATUS 0 Nov 04 22:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_36(8156159) done Nov 04 22:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_566(8157051) WEXITSTATUS 0 Nov 04 22:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_566(8157051) done Nov 04 22:16:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_656(8157162) WEXITSTATUS 0 Nov 04 22:16:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_656(8157162) done Nov 04 22:16:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_641(8157147) WEXITSTATUS 0 Nov 04 22:16:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_641(8157147) done Nov 04 22:16:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_502(8156987) WEXITSTATUS 0 Nov 04 22:16:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_502(8156987) done Nov 04 22:16:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_895(8157438) WEXITSTATUS 0 Nov 04 22:16:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_895(8157438) done Nov 04 22:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_420(8156885) WEXITSTATUS 0 Nov 04 22:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_420(8156885) done Nov 04 22:16:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_409(8156874) WEXITSTATUS 0 Nov 04 22:16:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_409(8156874) done Nov 04 22:16:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_500(8156985) WEXITSTATUS 0 Nov 04 22:16:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_500(8156985) done Nov 04 22:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_723(8157245) WEXITSTATUS 0 Nov 04 22:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_723(8157245) done Nov 04 22:16:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_707(8157229) WEXITSTATUS 0 Nov 04 22:16:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_707(8157229) done Nov 04 22:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_601(8157107) WEXITSTATUS 0 Nov 04 22:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_601(8157107) done Nov 04 22:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_680(8157186) WEXITSTATUS 0 Nov 04 22:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_680(8157186) done Nov 04 22:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2395(8152439) WEXITSTATUS 0 Nov 04 22:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2395(8152439) done Nov 04 22:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_503(8156988) WEXITSTATUS 0 Nov 04 22:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_503(8156988) done Nov 04 22:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_30(8156383) WEXITSTATUS 0 Nov 04 22:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_30(8156383) done Nov 04 22:16:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_521(8157006) WEXITSTATUS 0 Nov 04 22:16:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_521(8157006) done Nov 04 22:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_2(8156125) WEXITSTATUS 0 Nov 04 22:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_2(8156125) done Nov 04 22:16:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5028(8158324) WEXITSTATUS 1 Nov 04 22:16:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5028(8158324) done Nov 04 22:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157917 WEXITSTATUS 0 Nov 04 22:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157917 done Nov 04 22:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_889(8157432) WEXITSTATUS 0 Nov 04 22:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_889(8157432) done Nov 04 22:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_636(8157142) WEXITSTATUS 0 Nov 04 22:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_636(8157142) done Nov 04 22:16:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_682(8157188) WEXITSTATUS 0 Nov 04 22:16:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_682(8157188) done Nov 04 22:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157928 WEXITSTATUS 0 Nov 04 22:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157928 done Nov 04 22:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_34(8156157) WEXITSTATUS 0 Nov 04 22:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_34(8156157) done Nov 04 22:16:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_642(8157148) WEXITSTATUS 0 Nov 04 22:16:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_642(8157148) done Nov 04 22:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3766(8142955) WEXITSTATUS 1 Nov 04 22:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3766(8142955) done Nov 04 22:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_516(8157001) WEXITSTATUS 0 Nov 04 22:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_516(8157001) done Nov 04 22:16:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_764(8157286) WEXITSTATUS 0 Nov 04 22:16:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_764(8157286) done Nov 04 22:16:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_673(8157179) WEXITSTATUS 0 Nov 04 22:16:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_673(8157179) done Nov 04 22:16:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_517(8157002) WEXITSTATUS 0 Nov 04 22:16:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_517(8157002) done Nov 04 22:16:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_677(8157183) WEXITSTATUS 0 Nov 04 22:16:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_677(8157183) done Nov 04 22:16:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_890(8157433) WEXITSTATUS 0 Nov 04 22:16:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_890(8157433) done Nov 04 22:16:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_90(8156443) WEXITSTATUS 0 Nov 04 22:16:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_90(8156443) done Nov 04 22:16:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_721(8157243) WEXITSTATUS 0 Nov 04 22:16:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_721(8157243) done Nov 04 22:16:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_766(8157288) WEXITSTATUS 0 Nov 04 22:16:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_766(8157288) done Nov 04 22:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157894 WEXITSTATUS 0 Nov 04 22:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157894 done Nov 04 22:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157905 WEXITSTATUS 0 Nov 04 22:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157905 done Nov 04 22:17:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_686(8157192) WEXITSTATUS 0 Nov 04 22:17:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_686(8157192) done Nov 04 22:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_715(8157237) WEXITSTATUS 0 Nov 04 22:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_715(8157237) done Nov 04 22:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157907 WEXITSTATUS 0 Nov 04 22:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157907 done Nov 04 22:17:05 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 04 22:17:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 04 22:17:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 04 22:17:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_685(8157191) WEXITSTATUS 0 Nov 04 22:17:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_685(8157191) done Nov 04 22:17:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157893 WEXITSTATUS 0 Nov 04 22:17:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157893 done Nov 04 22:17:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_515(8157000) WEXITSTATUS 0 Nov 04 22:17:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_515(8157000) done Nov 04 22:17:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3535(8158455) in gpu on r4513u23n01 Nov 04 22:17:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3536(8158456) in gpu on r903u34n01 Nov 04 22:17:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3537(8158457) in gpu on r903u34n01 Nov 04 22:17:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1820(8158458) in day on r904u10n01 Nov 04 22:17:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1821(8158459) in day on r904u10n03 Nov 04 22:17:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1822(8158460) in day on r914u01n04 Nov 04 22:17:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 22:17:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157510 in scavenge on r816u09n10 Nov 04 22:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_513(8156998) WEXITSTATUS 0 Nov 04 22:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_513(8156998) done Nov 04 22:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_678(8157184) WEXITSTATUS 0 Nov 04 22:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_678(8157184) done Nov 04 22:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_672(8157178) WEXITSTATUS 0 Nov 04 22:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_672(8157178) done Nov 04 22:17:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2020(8151580) WEXITSTATUS 0 Nov 04 22:17:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2020(8151580) done Nov 04 22:17:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_776(8157298) WEXITSTATUS 0 Nov 04 22:17:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_776(8157298) done Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5121(8158461) in scavenge on r801u23n01 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5122(8158462) in scavenge on r802u06n03 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5123(8158463) in scavenge on r805u27n04 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5124(8158464) in scavenge on r806u14n03 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5125(8158465) in scavenge on r806u14n03 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5126(8158466) in scavenge on r807u08n03 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5127(8158467) in scavenge on r909u29n02 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5128(8158468) in scavenge on r913u05n04 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5129(8158469) in scavenge on r913u13n01 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5130(8158470) in scavenge on r904u10n01 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5131(8158471) in scavenge on r904u10n01 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5132(8158472) in scavenge on r904u10n01 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5133(8158473) in scavenge on r904u10n03 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5134(8158474) in scavenge on r815u15n02 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5135(8158475) in scavenge on r908u24n02 Nov 04 22:17:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5136(8158476) in scavenge on r806u41n01 Nov 04 22:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157924 WEXITSTATUS 0 Nov 04 22:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157924 done Nov 04 22:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4821(8157787) WEXITSTATUS 1 Nov 04 22:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4821(8157787) done Nov 04 22:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_700(8157222) WEXITSTATUS 0 Nov 04 22:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_700(8157222) done Nov 04 22:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_719(8157241) WEXITSTATUS 0 Nov 04 22:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_719(8157241) done Nov 04 22:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_888(8157431) WEXITSTATUS 0 Nov 04 22:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_888(8157431) done Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157906 WEXITSTATUS 0 Nov 04 22:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157906 done Nov 04 22:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_757(8157279) WEXITSTATUS 0 Nov 04 22:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_757(8157279) done Nov 04 22:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_894(8157437) WEXITSTATUS 0 Nov 04 22:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_894(8157437) done Nov 04 22:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_717(8157239) WEXITSTATUS 0 Nov 04 22:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_717(8157239) done Nov 04 22:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_544(8157029) WEXITSTATUS 0 Nov 04 22:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_544(8157029) done Nov 04 22:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_548(8157033) WEXITSTATUS 0 Nov 04 22:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_548(8157033) done Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_168(7572336) Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_320(7572488) Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_374(7572542) Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_379(7572547) Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_449(7572617) Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_233(8114499) Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_366(8114633) Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_48(8148895) Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_748(8157270) WEXITSTATUS 0 Nov 04 22:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_748(8157270) done Nov 04 22:17:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_775(8157297) WEXITSTATUS 0 Nov 04 22:17:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_775(8157297) done Nov 04 22:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_898(8157441) WEXITSTATUS 0 Nov 04 22:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_898(8157441) done Nov 04 22:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3533(8158424) WEXITSTATUS 0 Nov 04 22:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3533(8158424) done Nov 04 22:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_706(8157228) WEXITSTATUS 0 Nov 04 22:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_706(8157228) done Nov 04 22:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_43(8156166) WEXITSTATUS 0 Nov 04 22:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_43(8156166) done Nov 04 22:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_602(8157108) WEXITSTATUS 0 Nov 04 22:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_602(8157108) done Nov 04 22:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_859(8157402) WEXITSTATUS 0 Nov 04 22:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_859(8157402) done Nov 04 22:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_501(8156986) WEXITSTATUS 0 Nov 04 22:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_501(8156986) done Nov 04 22:17:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_37(8156160) WEXITSTATUS 0 Nov 04 22:17:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_37(8156160) done Nov 04 22:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_518(8157003) WEXITSTATUS 0 Nov 04 22:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_518(8157003) done Nov 04 22:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_17(8156140) WEXITSTATUS 0 Nov 04 22:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_17(8156140) done Nov 04 22:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_31(8156154) WEXITSTATUS 0 Nov 04 22:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_31(8156154) done Nov 04 22:17:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157925 WEXITSTATUS 0 Nov 04 22:17:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157925 done Nov 04 22:17:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_705(8157227) WEXITSTATUS 0 Nov 04 22:17:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_705(8157227) done Nov 04 22:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_676(8157182) WEXITSTATUS 0 Nov 04 22:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_676(8157182) done Nov 04 22:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_770(8157292) WEXITSTATUS 0 Nov 04 22:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_770(8157292) done Nov 04 22:17:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_720(8157242) WEXITSTATUS 0 Nov 04 22:17:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_720(8157242) done Nov 04 22:17:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157922 WEXITSTATUS 0 Nov 04 22:17:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157922 done Nov 04 22:17:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_703(8157225) WEXITSTATUS 0 Nov 04 22:17:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_703(8157225) done Nov 04 22:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4682(8156344) WEXITSTATUS 1 Nov 04 22:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4682(8156344) done Nov 04 22:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_754(8157276) WEXITSTATUS 0 Nov 04 22:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_754(8157276) done Nov 04 22:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157895 WEXITSTATUS 0 Nov 04 22:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157895 done Nov 04 22:17:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_4(8156127) WEXITSTATUS 0 Nov 04 22:17:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_4(8156127) done Nov 04 22:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_704(8157226) WEXITSTATUS 0 Nov 04 22:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_704(8157226) done Nov 04 22:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157915 WEXITSTATUS 0 Nov 04 22:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157915 done Nov 04 22:17:59 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2819(8128404) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_652(8157158) WEXITSTATUS 0 Nov 04 22:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_652(8157158) done Nov 04 22:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_772(8157294) WEXITSTATUS 0 Nov 04 22:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_772(8157294) done Nov 04 22:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_21(8156144) WEXITSTATUS 0 Nov 04 22:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_21(8156144) done Nov 04 22:18:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2819(8128404) Nov 04 22:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_750(8157272) WEXITSTATUS 0 Nov 04 22:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_750(8157272) done Nov 04 22:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_768(8157290) WEXITSTATUS 0 Nov 04 22:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_768(8157290) done Nov 04 22:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157913 WEXITSTATUS 0 Nov 04 22:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157913 done Nov 04 22:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_709(8157231) WEXITSTATUS 0 Nov 04 22:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_709(8157231) done Nov 04 22:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_897(8157440) WEXITSTATUS 0 Nov 04 22:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_897(8157440) done Nov 04 22:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4666(8156328) WEXITSTATUS 1 Nov 04 22:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4666(8156328) done Nov 04 22:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_675(8157181) WEXITSTATUS 0 Nov 04 22:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_675(8157181) done Nov 04 22:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157931 WEXITSTATUS 0 Nov 04 22:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157931 done Nov 04 22:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_654(8157160) WEXITSTATUS 0 Nov 04 22:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_654(8157160) done Nov 04 22:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_716(8157238) WEXITSTATUS 0 Nov 04 22:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_716(8157238) done Nov 04 22:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_777(8157299) WEXITSTATUS 0 Nov 04 22:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_777(8157299) done Nov 04 22:18:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_33(8156156) WEXITSTATUS 0 Nov 04 22:18:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_33(8156156) done Nov 04 22:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_546(8157031) WEXITSTATUS 0 Nov 04 22:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_546(8157031) done Nov 04 22:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_759(8157281) WEXITSTATUS 0 Nov 04 22:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_759(8157281) done Nov 04 22:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_760(8157282) WEXITSTATUS 0 Nov 04 22:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_760(8157282) done Nov 04 22:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_659(8157165) WEXITSTATUS 0 Nov 04 22:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_659(8157165) done Nov 04 22:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_718(8157240) WEXITSTATUS 0 Nov 04 22:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_718(8157240) done Nov 04 22:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_674(8157180) WEXITSTATUS 0 Nov 04 22:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_674(8157180) done Nov 04 22:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_547(8157032) WEXITSTATUS 0 Nov 04 22:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_547(8157032) done Nov 04 22:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_899(8157442) WEXITSTATUS 0 Nov 04 22:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_899(8157442) done Nov 04 22:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_543(8157028) WEXITSTATUS 0 Nov 04 22:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_543(8157028) done Nov 04 22:18:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_22(8156145) WEXITSTATUS 0 Nov 04 22:18:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_22(8156145) done Nov 04 22:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_751(8157273) WEXITSTATUS 0 Nov 04 22:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_751(8157273) done Nov 04 22:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_763(8157285) WEXITSTATUS 0 Nov 04 22:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_763(8157285) done Nov 04 22:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_774(8157296) WEXITSTATUS 0 Nov 04 22:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_774(8157296) done Nov 04 22:18:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157930 WEXITSTATUS 0 Nov 04 22:18:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157930 done Nov 04 22:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_752(8157274) WEXITSTATUS 0 Nov 04 22:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_752(8157274) done Nov 04 22:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_711(8157233) WEXITSTATUS 0 Nov 04 22:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_711(8157233) done Nov 04 22:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_14(8156137) WEXITSTATUS 0 Nov 04 22:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_14(8156137) done Nov 04 22:18:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5012(8158296) WEXITSTATUS 1 Nov 04 22:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5012(8158296) done Nov 04 22:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_18(8156141) WEXITSTATUS 0 Nov 04 22:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_18(8156141) done Nov 04 22:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_666(8157172) WEXITSTATUS 0 Nov 04 22:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_666(8157172) done Nov 04 22:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_32(8156155) WEXITSTATUS 0 Nov 04 22:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_32(8156155) done Nov 04 22:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_761(8157283) WEXITSTATUS 0 Nov 04 22:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_761(8157283) done Nov 04 22:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_997(8157651) WEXITSTATUS 0 Nov 04 22:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_997(8157651) done Nov 04 22:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_712(8157234) WEXITSTATUS 0 Nov 04 22:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_712(8157234) done Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_722(8157244) WEXITSTATUS 0 Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_722(8157244) done Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_545(8157030) WEXITSTATUS 0 Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_545(8157030) done Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_15(8156138) WEXITSTATUS 0 Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_15(8156138) done Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_542(8157027) WEXITSTATUS 0 Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_542(8157027) done Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_27(8156150) WEXITSTATUS 0 Nov 04 22:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_27(8156150) done Nov 04 22:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_714(8157236) WEXITSTATUS 0 Nov 04 22:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_714(8157236) done Nov 04 22:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157899 WEXITSTATUS 0 Nov 04 22:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157899 done Nov 04 22:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4620(8156204) WEXITSTATUS 1 Nov 04 22:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4620(8156204) done Nov 04 22:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_771(8157293) WEXITSTATUS 0 Nov 04 22:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_771(8157293) done Nov 04 22:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5134(8158474) WEXITSTATUS 1 Nov 04 22:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5134(8158474) done Nov 04 22:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_773(8157295) WEXITSTATUS 0 Nov 04 22:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_773(8157295) done Nov 04 22:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_755(8157277) WEXITSTATUS 0 Nov 04 22:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_755(8157277) done Nov 04 22:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_758(8157280) WEXITSTATUS 0 Nov 04 22:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_758(8157280) done Nov 04 22:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157896 WEXITSTATUS 0 Nov 04 22:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157896 done Nov 04 22:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_756(8157278) WEXITSTATUS 0 Nov 04 22:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_756(8157278) done Nov 04 22:19:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_651(8157157) WEXITSTATUS 0 Nov 04 22:19:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_651(8157157) done Nov 04 22:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_767(8157289) WEXITSTATUS 0 Nov 04 22:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_767(8157289) done Nov 04 22:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3536(8158456) WEXITSTATUS 0 Nov 04 22:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3536(8158456) done Nov 04 22:19:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157901 WEXITSTATUS 0 Nov 04 22:19:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157901 done Nov 04 22:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_466(8156931) WEXITSTATUS 0 Nov 04 22:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_466(8156931) done Nov 04 22:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157884 WEXITSTATUS 0 Nov 04 22:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157884 done Nov 04 22:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_769(8157291) WEXITSTATUS 0 Nov 04 22:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_769(8157291) done Nov 04 22:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_549(8157034) WEXITSTATUS 0 Nov 04 22:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_549(8157034) done Nov 04 22:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_838(8157381) WEXITSTATUS 0 Nov 04 22:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_838(8157381) done Nov 04 22:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_837(8157380) WEXITSTATUS 0 Nov 04 22:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_837(8157380) done Nov 04 22:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_829(8157372) WEXITSTATUS 0 Nov 04 22:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_829(8157372) done Nov 04 22:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5080(8158392) WEXITSTATUS 1 Nov 04 22:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5080(8158392) done Nov 04 22:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_735(8157257) WEXITSTATUS 0 Nov 04 22:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_735(8157257) done Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_816(8157359) WEXITSTATUS 0 Nov 04 22:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_816(8157359) done Nov 04 22:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_753(8157275) WEXITSTATUS 0 Nov 04 22:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_753(8157275) done Nov 04 22:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1102(8131594) WEXITSTATUS 0 Nov 04 22:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1102(8131594) done Nov 04 22:19:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_10(8156133) WEXITSTATUS 0 Nov 04 22:19:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_10(8156133) done Nov 04 22:19:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_29(8156152) WEXITSTATUS 0 Nov 04 22:19:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_29(8156152) done Nov 04 22:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157911 WEXITSTATUS 0 Nov 04 22:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157911 done Nov 04 22:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_746(8157268) WEXITSTATUS 0 Nov 04 22:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_746(8157268) done Nov 04 22:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_661(8157167) WEXITSTATUS 0 Nov 04 22:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_661(8157167) done Nov 04 22:19:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_669(8157175) WEXITSTATUS 0 Nov 04 22:19:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_669(8157175) done Nov 04 22:19:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_701(8157223) WEXITSTATUS 0 Nov 04 22:19:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_701(8157223) done Nov 04 22:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157916 WEXITSTATUS 0 Nov 04 22:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157916 done Nov 04 22:19:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_834(8157377) WEXITSTATUS 0 Nov 04 22:19:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_834(8157377) done Nov 04 22:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_35(8156158) WEXITSTATUS 0 Nov 04 22:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_35(8156158) done Nov 04 22:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4801(8157697) WEXITSTATUS 1 Nov 04 22:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4801(8157697) done Nov 04 22:19:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3538(8158477) in gpu on r808u17n01 Nov 04 22:19:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3539(8158478) in gpu on r903u34n01 Nov 04 22:19:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1823(8158479) in day on r807u05n04 Nov 04 22:19:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1824(8158480) in day on r909u10n04 Nov 04 22:19:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1825(8158481) in day on r806u02n04 Nov 04 22:19:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1826(8158482) in day on r806u06n03 Nov 04 22:19:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1827(8158483) in day on r806u23n02 Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_749(8157271) WEXITSTATUS 0 Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_749(8157271) done Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_300(7572468) Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_636(7572805) Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_676(7572845) Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_959(7573128) Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8124758 Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_708(8157230) WEXITSTATUS 0 Nov 04 22:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_708(8157230) done Nov 04 22:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_710(8157232) WEXITSTATUS 0 Nov 04 22:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_710(8157232) done Nov 04 22:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_765(8157287) WEXITSTATUS 0 Nov 04 22:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_765(8157287) done Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5137(8158484) in scavenge on r801u06n02 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5138(8158485) in scavenge on r805u27n04 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5139(8158486) in scavenge on r807u05n04 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5140(8158487) in scavenge on r909u13n02 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5141(8158488) in scavenge on r904u10n03 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5142(8158489) in scavenge on r904u10n03 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5143(8158490) in scavenge on r912u36n02 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5144(8158491) in scavenge on r913u01n02 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5145(8158492) in scavenge on r913u05n04 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5146(8158493) in scavenge on r919u09n01 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5147(8158494) in scavenge on r919u09n03 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5148(8158495) in scavenge on r919u36n04 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5149(8158496) in scavenge on r806u04n03 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5150(8158497) in scavenge on r806u04n04 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5151(8158498) in scavenge on r806u06n03 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5152(8158499) in scavenge on r806u06n03 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5153(8158500) in scavenge on r806u06n03 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5154(8158501) in scavenge on r806u23n02 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5155(8158502) in scavenge on r806u23n02 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5156(8158503) in scavenge on r815u15n02 Nov 04 22:20:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5157(8158504) in scavenge on r908u24n02 Nov 04 22:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_670(8157176) WEXITSTATUS 0 Nov 04 22:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_670(8157176) done Nov 04 22:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_11(8156134) WEXITSTATUS 0 Nov 04 22:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_11(8156134) done Nov 04 22:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_874(8157417) WEXITSTATUS 0 Nov 04 22:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_874(8157417) done Nov 04 22:20:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_830(8157373) WEXITSTATUS 0 Nov 04 22:20:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_830(8157373) done Nov 04 22:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_831(8157374) WEXITSTATUS 0 Nov 04 22:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_831(8157374) done Nov 04 22:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_999(8156351) WEXITSTATUS 0 Nov 04 22:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_999(8156351) done Nov 04 22:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_822(8157365) WEXITSTATUS 0 Nov 04 22:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_822(8157365) done Nov 04 22:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_815(8157358) WEXITSTATUS 0 Nov 04 22:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_815(8157358) done Nov 04 22:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_664(8157170) WEXITSTATUS 0 Nov 04 22:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_664(8157170) done Nov 04 22:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157909 WEXITSTATUS 0 Nov 04 22:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157909 done Nov 04 22:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_663(8157169) WEXITSTATUS 0 Nov 04 22:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_663(8157169) done Nov 04 22:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_996(8157650) WEXITSTATUS 0 Nov 04 22:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_996(8157650) done Nov 04 22:20:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_459(8156924) WEXITSTATUS 0 Nov 04 22:20:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_459(8156924) done Nov 04 22:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_778(8157300) WEXITSTATUS 0 Nov 04 22:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_778(8157300) done Nov 04 22:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_998(8157652) WEXITSTATUS 0 Nov 04 22:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_998(8157652) done Nov 04 22:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_730(8157252) WEXITSTATUS 0 Nov 04 22:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_730(8157252) done Nov 04 22:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_743(8157265) WEXITSTATUS 0 Nov 04 22:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_743(8157265) done Nov 04 22:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_740(8157262) WEXITSTATUS 0 Nov 04 22:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_740(8157262) done Nov 04 22:20:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_825(8157368) WEXITSTATUS 0 Nov 04 22:20:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_825(8157368) done Nov 04 22:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_745(8157267) WEXITSTATUS 0 Nov 04 22:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_745(8157267) done Nov 04 22:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_796(8157318) WEXITSTATUS 0 Nov 04 22:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_796(8157318) done Nov 04 22:20:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_736(8157258) WEXITSTATUS 0 Nov 04 22:20:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_736(8157258) done Nov 04 22:20:30 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1843(8124770) Nov 04 22:20:30 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8144813_49(8144813) Nov 04 22:20:30 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8149757_48(8156197) had to be killed Nov 04 22:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_246(8156654) WEXITSTATUS 0 Nov 04 22:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_246(8156654) done Nov 04 22:20:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_684(8157190) WEXITSTATUS 0 Nov 04 22:20:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_684(8157190) done Nov 04 22:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_665(8157171) WEXITSTATUS 0 Nov 04 22:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_665(8157171) done Nov 04 22:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3539(8158478) WEXITSTATUS 0 Nov 04 22:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3539(8158478) done Nov 04 22:20:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_3(8156126) WEXITSTATUS 0 Nov 04 22:20:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_3(8156126) done Nov 04 22:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_23(8156146) WEXITSTATUS 0 Nov 04 22:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_23(8156146) done Nov 04 22:20:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_648(8157154) WEXITSTATUS 0 Nov 04 22:20:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_648(8157154) done Nov 04 22:20:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_819(8157362) WEXITSTATUS 0 Nov 04 22:20:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_819(8157362) done Nov 04 22:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_833(8157376) WEXITSTATUS 0 Nov 04 22:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_833(8157376) done Nov 04 22:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3976(8146125) WEXITSTATUS 1 Nov 04 22:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3976(8146125) done Nov 04 22:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_922(8157576) WEXITSTATUS 0 Nov 04 22:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_922(8157576) done Nov 04 22:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_731(8157253) WEXITSTATUS 0 Nov 04 22:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_731(8157253) done Nov 04 22:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_8(8156131) WEXITSTATUS 0 Nov 04 22:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_8(8156131) done Nov 04 22:20:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_875(8157418) WEXITSTATUS 0 Nov 04 22:20:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_875(8157418) done Nov 04 22:20:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_662(8157168) WEXITSTATUS 0 Nov 04 22:20:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_662(8157168) done Nov 04 22:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_650(8157156) WEXITSTATUS 0 Nov 04 22:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_650(8157156) done Nov 04 22:20:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_643(8157149) WEXITSTATUS 0 Nov 04 22:20:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_643(8157149) done Nov 04 22:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_733(8157255) WEXITSTATUS 0 Nov 04 22:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_733(8157255) done Nov 04 22:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_762(8157284) WEXITSTATUS 0 Nov 04 22:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_762(8157284) done Nov 04 22:20:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_926(8157580) WEXITSTATUS 0 Nov 04 22:20:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_926(8157580) done Nov 04 22:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_826(8157369) WEXITSTATUS 0 Nov 04 22:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_826(8157369) done Nov 04 22:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2216(8152008) WEXITSTATUS 0 Nov 04 22:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2216(8152008) done Nov 04 22:21:00 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8142116 Nov 04 22:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_646(8157152) WEXITSTATUS 0 Nov 04 22:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_646(8157152) done Nov 04 22:21:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1152(8132730) WEXITSTATUS 0 Nov 04 22:21:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1152(8132730) done Nov 04 22:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_702(8157224) WEXITSTATUS 0 Nov 04 22:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_702(8157224) done Nov 04 22:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_667(8157173) WEXITSTATUS 0 Nov 04 22:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_667(8157173) done Nov 04 22:21:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_824(8157367) WEXITSTATUS 0 Nov 04 22:21:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_824(8157367) done Nov 04 22:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_835(8157378) WEXITSTATUS 0 Nov 04 22:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_835(8157378) done Nov 04 22:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_24(8156147) WEXITSTATUS 0 Nov 04 22:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_24(8156147) done Nov 04 22:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_742(8157264) WEXITSTATUS 0 Nov 04 22:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_742(8157264) done Nov 04 22:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4283(8154049) WEXITSTATUS 1 Nov 04 22:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4283(8154049) done Nov 04 22:21:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_817(8157360) WEXITSTATUS 0 Nov 04 22:21:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_817(8157360) done Nov 04 22:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_876(8157419) WEXITSTATUS 0 Nov 04 22:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_876(8157419) done Nov 04 22:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_713(8157235) WEXITSTATUS 0 Nov 04 22:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_713(8157235) done Nov 04 22:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_649(8157155) WEXITSTATUS 0 Nov 04 22:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_649(8157155) done Nov 04 22:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_732(8157254) WEXITSTATUS 0 Nov 04 22:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_732(8157254) done Nov 04 22:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_878(8157421) WEXITSTATUS 0 Nov 04 22:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_878(8157421) done Nov 04 22:21:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_821(8157364) WEXITSTATUS 0 Nov 04 22:21:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_821(8157364) done Nov 04 22:21:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_739(8157261) WEXITSTATUS 0 Nov 04 22:21:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_739(8157261) done Nov 04 22:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5056(8158361) WEXITSTATUS 1 Nov 04 22:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5056(8158361) done Nov 04 22:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_20(8156143) WEXITSTATUS 0 Nov 04 22:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_20(8156143) done Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_7(8156130) WEXITSTATUS 0 Nov 04 22:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_7(8156130) done Nov 04 22:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_832(8157375) WEXITSTATUS 0 Nov 04 22:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_832(8157375) done Nov 04 22:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_644(8157150) WEXITSTATUS 0 Nov 04 22:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_644(8157150) done Nov 04 22:21:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_836(8157379) WEXITSTATUS 0 Nov 04 22:21:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_836(8157379) done Nov 04 22:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_44(8156167) WEXITSTATUS 0 Nov 04 22:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_44(8156167) done Nov 04 22:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3537(8158457) WEXITSTATUS 0 Nov 04 22:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3537(8158457) done Nov 04 22:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4460(8155777) WEXITSTATUS 1 Nov 04 22:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4460(8155777) done Nov 04 22:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4265(8153360) WEXITSTATUS 1 Nov 04 22:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4265(8153360) done Nov 04 22:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5093(8158417) WEXITSTATUS 1 Nov 04 22:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5093(8158417) done Nov 04 22:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_877(8157420) WEXITSTATUS 0 Nov 04 22:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_877(8157420) done Nov 04 22:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_823(8157366) WEXITSTATUS 0 Nov 04 22:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_823(8157366) done Nov 04 22:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_820(8157363) WEXITSTATUS 0 Nov 04 22:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_820(8157363) done Nov 04 22:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_828(8157371) WEXITSTATUS 0 Nov 04 22:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_828(8157371) done Nov 04 22:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_946(8157600) WEXITSTATUS 0 Nov 04 22:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_946(8157600) done Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_806(8157349) WEXITSTATUS 0 Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_806(8157349) done Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_801(8157344) WEXITSTATUS 0 Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_801(8157344) done Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_810(8157353) WEXITSTATUS 0 Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_810(8157353) done Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_804(8157347) WEXITSTATUS 0 Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_804(8157347) done Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_944(8157598) WEXITSTATUS 0 Nov 04 22:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_944(8157598) done Nov 04 22:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_814(8157357) WEXITSTATUS 0 Nov 04 22:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_814(8157357) done Nov 04 22:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_738(8157260) WEXITSTATUS 0 Nov 04 22:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_738(8157260) done Nov 04 22:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_945(8157599) WEXITSTATUS 0 Nov 04 22:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_945(8157599) done Nov 04 22:22:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_695(8157201) WEXITSTATUS 0 Nov 04 22:22:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_695(8157201) done Nov 04 22:22:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4785(8157660) WEXITSTATUS 1 Nov 04 22:22:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4785(8157660) done Nov 04 22:22:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1865(8151191) WEXITSTATUS 0 Nov 04 22:22:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1865(8151191) done Nov 04 22:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_860(8157403) WEXITSTATUS 0 Nov 04 22:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_860(8157403) done Nov 04 22:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_729(8157251) WEXITSTATUS 0 Nov 04 22:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_729(8157251) done Nov 04 22:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157511 NodeList=r816u35n12 #CPUs=12 Partition=scavenge Nov 04 22:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_943(8157597) WEXITSTATUS 0 Nov 04 22:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_943(8157597) done Nov 04 22:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_645(8157151) WEXITSTATUS 0 Nov 04 22:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_645(8157151) done Nov 04 22:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_693(8157199) WEXITSTATUS 0 Nov 04 22:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_693(8157199) done Nov 04 22:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_734(8157256) WEXITSTATUS 0 Nov 04 22:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_734(8157256) done Nov 04 22:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157885 WEXITSTATUS 0 Nov 04 22:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157885 done Nov 04 22:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_5(8156128) WEXITSTATUS 0 Nov 04 22:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_5(8156128) done Nov 04 22:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_737(8157259) WEXITSTATUS 0 Nov 04 22:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_737(8157259) done Nov 04 22:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_947(8157601) WEXITSTATUS 0 Nov 04 22:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_947(8157601) done Nov 04 22:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5117(8158450) WEXITSTATUS 1 Nov 04 22:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5117(8158450) done Nov 04 22:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_949(8157603) WEXITSTATUS 0 Nov 04 22:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_949(8157603) done Nov 04 22:22:30 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_935(8115204) Nov 04 22:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_696(8157202) WEXITSTATUS 0 Nov 04 22:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_696(8157202) done Nov 04 22:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_932(8157586) WEXITSTATUS 0 Nov 04 22:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_932(8157586) done Nov 04 22:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_827(8157370) WEXITSTATUS 0 Nov 04 22:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_827(8157370) done Nov 04 22:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_799(8157321) WEXITSTATUS 0 Nov 04 22:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_799(8157321) done Nov 04 22:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_697(8157203) WEXITSTATUS 0 Nov 04 22:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_697(8157203) done Nov 04 22:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_690(8157196) WEXITSTATUS 0 Nov 04 22:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_690(8157196) done Nov 04 22:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_744(8157266) WEXITSTATUS 0 Nov 04 22:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_744(8157266) done Nov 04 22:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_808(8157351) WEXITSTATUS 0 Nov 04 22:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_808(8157351) done Nov 04 22:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_726(8157248) WEXITSTATUS 0 Nov 04 22:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_726(8157248) done Nov 04 22:22:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_784(8157306) WEXITSTATUS 0 Nov 04 22:22:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_784(8157306) done Nov 04 22:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_939(8157593) WEXITSTATUS 0 Nov 04 22:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_939(8157593) done Nov 04 22:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3540(8158505) in gpu on r4513u26n01 Nov 04 22:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3541(8158506) in gpu on r903u34n01 Nov 04 22:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1828(8158507) in day on r807u08n02 Nov 04 22:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1829(8158508) in day on r806u02n03 Nov 04 22:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1830(8158509) in day on r806u06n02 Nov 04 22:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1831(8158510) in day on r909u04n04 Nov 04 22:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1832(8158511) in day on r909u29n04 Nov 04 22:22:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 22:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_270(7572438) WEXITSTATUS 0 Nov 04 22:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_270(7572438) done Nov 04 22:22:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_698(8157204) WEXITSTATUS 0 Nov 04 22:22:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_698(8157204) done Nov 04 22:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_687(8157193) WEXITSTATUS 0 Nov 04 22:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_687(8157193) done Nov 04 22:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_668(8157174) WEXITSTATUS 0 Nov 04 22:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_668(8157174) done Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5158(8158512) in scavenge on r802u06n02 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5159(8158513) in scavenge on r912u01n01 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5160(8158514) in scavenge on r917u07n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5161(8158515) in scavenge on r903u34n01 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5162(8158516) in scavenge on r907u28n01 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5163(8158517) in scavenge on r807u05n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5164(8158518) in scavenge on r912u32n03 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5165(8158519) in scavenge on r913u28n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5166(8158520) in scavenge on r913u28n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5167(8158521) in scavenge on r805u02n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5168(8158522) in scavenge on r805u04n02 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5169(8158523) in scavenge on r805u09n01 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5170(8158524) in scavenge on r806u02n03 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5171(8158525) in scavenge on r806u02n03 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5172(8158526) in scavenge on r806u02n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5173(8158527) in scavenge on r806u02n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5174(8158528) in scavenge on r806u04n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5175(8158529) in scavenge on r806u04n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5176(8158530) in scavenge on r909u04n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5177(8158531) in scavenge on r909u04n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5178(8158532) in scavenge on r909u04n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5179(8158533) in scavenge on r909u04n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5180(8158534) in scavenge on r909u10n02 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5181(8158535) in scavenge on r909u29n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5182(8158536) in scavenge on r909u29n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5183(8158537) in scavenge on r909u29n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5184(8158538) in scavenge on r909u29n04 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5185(8158539) in scavenge on r815u15n03 Nov 04 22:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5186(8158540) in scavenge on r806u31n01 Nov 04 22:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_647(8157153) WEXITSTATUS 0 Nov 04 22:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_647(8157153) done Nov 04 22:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_791(8157313) WEXITSTATUS 0 Nov 04 22:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_791(8157313) done Nov 04 22:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_948(8157602) WEXITSTATUS 0 Nov 04 22:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_948(8157602) done Nov 04 22:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3528(8158269) WEXITSTATUS 0 Nov 04 22:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3528(8158269) done Nov 04 22:23:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_871(8127985) in scavenge on r806u28n01 Nov 04 22:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_725(8157247) WEXITSTATUS 0 Nov 04 22:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_725(8157247) done Nov 04 22:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_802(8157345) WEXITSTATUS 0 Nov 04 22:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_802(8157345) done Nov 04 22:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_793(8157315) WEXITSTATUS 0 Nov 04 22:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_793(8157315) done Nov 04 22:23:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_46(8156169) WEXITSTATUS 0 Nov 04 22:23:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_46(8156169) done Nov 04 22:23:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5126(8158466) WEXITSTATUS 1 Nov 04 22:23:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5126(8158466) done Nov 04 22:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_805(8157348) WEXITSTATUS 0 Nov 04 22:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_805(8157348) done Nov 04 22:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_694(8157200) WEXITSTATUS 0 Nov 04 22:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_694(8157200) done Nov 04 22:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_854(8157397) WEXITSTATUS 0 Nov 04 22:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_854(8157397) done Nov 04 22:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2267(8152184) WEXITSTATUS 0 Nov 04 22:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2267(8152184) done Nov 04 22:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_699(8157205) WEXITSTATUS 0 Nov 04 22:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_699(8157205) done Nov 04 22:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_811(8157354) WEXITSTATUS 0 Nov 04 22:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_811(8157354) done Nov 04 22:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_807(8157350) WEXITSTATUS 0 Nov 04 22:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_807(8157350) done Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_795(8157317) WEXITSTATUS 0 Nov 04 22:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_795(8157317) done Nov 04 22:23:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:23:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:23:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_792(8157314) WEXITSTATUS 0 Nov 04 22:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_792(8157314) done Nov 04 22:23:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1850(8124782) Nov 04 22:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_803(8157346) WEXITSTATUS 0 Nov 04 22:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_803(8157346) done Nov 04 22:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_741(8157263) WEXITSTATUS 0 Nov 04 22:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_741(8157263) done Nov 04 22:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_935(8157589) WEXITSTATUS 0 Nov 04 22:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_935(8157589) done Nov 04 22:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_937(8157591) WEXITSTATUS 0 Nov 04 22:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_937(8157591) done Nov 04 22:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_794(8157316) WEXITSTATUS 0 Nov 04 22:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_794(8157316) done Nov 04 22:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_936(8157590) WEXITSTATUS 0 Nov 04 22:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_936(8157590) done Nov 04 22:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_844(8157387) WEXITSTATUS 0 Nov 04 22:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_844(8157387) done Nov 04 22:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_990(8157644) WEXITSTATUS 0 Nov 04 22:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_990(8157644) done Nov 04 22:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_839(8157382) WEXITSTATUS 0 Nov 04 22:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_839(8157382) done Nov 04 22:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_842(8157385) WEXITSTATUS 0 Nov 04 22:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_842(8157385) done Nov 04 22:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_40(8156163) WEXITSTATUS 0 Nov 04 22:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_40(8156163) done Nov 04 22:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_930(8157584) WEXITSTATUS 0 Nov 04 22:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_930(8157584) done Nov 04 22:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_931(8157585) WEXITSTATUS 0 Nov 04 22:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_931(8157585) done Nov 04 22:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4943(8158188) WEXITSTATUS 1 Nov 04 22:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4943(8158188) done Nov 04 22:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_927(8157581) WEXITSTATUS 0 Nov 04 22:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_927(8157581) done Nov 04 22:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_904(8157558) WEXITSTATUS 0 Nov 04 22:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_904(8157558) done Nov 04 22:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_917(8157571) WEXITSTATUS 0 Nov 04 22:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_917(8157571) done Nov 04 22:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_900(8157554) WEXITSTATUS 0 Nov 04 22:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_900(8157554) done Nov 04 22:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135380 WEXITSTATUS 0 Nov 04 22:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135380 done Nov 04 22:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_929(8157583) WEXITSTATUS 0 Nov 04 22:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_929(8157583) done Nov 04 22:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3541(8158506) WEXITSTATUS 0 Nov 04 22:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3541(8158506) done Nov 04 22:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_787(8157309) WEXITSTATUS 0 Nov 04 22:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_787(8157309) done Nov 04 22:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_688(8157194) WEXITSTATUS 0 Nov 04 22:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_688(8157194) done Nov 04 22:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_912(8157566) WEXITSTATUS 0 Nov 04 22:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_912(8157566) done Nov 04 22:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_845(8157388) WEXITSTATUS 0 Nov 04 22:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_845(8157388) done Nov 04 22:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_852(8157395) WEXITSTATUS 0 Nov 04 22:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_852(8157395) done Nov 04 22:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_691(8157197) WEXITSTATUS 0 Nov 04 22:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_691(8157197) done Nov 04 22:24:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_843(8157386) WEXITSTATUS 0 Nov 04 22:24:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_843(8157386) done Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5087(8158411) WEXITSTATUS 1 Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5087(8158411) done Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_788(8157310) WEXITSTATUS 0 Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_788(8157310) done Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_781(8157303) WEXITSTATUS 0 Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_781(8157303) done Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_812(8157355) WEXITSTATUS 0 Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_812(8157355) done Nov 04 22:24:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_916(8157570) WEXITSTATUS 0 Nov 04 22:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_916(8157570) done Nov 04 22:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5186(8158540) WEXITSTATUS 1 Nov 04 22:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5186(8158540) done Nov 04 22:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_989(8157643) WEXITSTATUS 0 Nov 04 22:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_989(8157643) done Nov 04 22:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_724(8157246) WEXITSTATUS 0 Nov 04 22:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_724(8157246) done Nov 04 22:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_988(8157642) WEXITSTATUS 0 Nov 04 22:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_988(8157642) done Nov 04 22:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_910(8157564) WEXITSTATUS 0 Nov 04 22:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_910(8157564) done Nov 04 22:24:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_934(8157588) WEXITSTATUS 0 Nov 04 22:24:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_934(8157588) done Nov 04 22:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_789(8157311) WEXITSTATUS 0 Nov 04 22:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_789(8157311) done Nov 04 22:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_797(8157319) WEXITSTATUS 0 Nov 04 22:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_797(8157319) done Nov 04 22:24:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157512 NodeList=r816u35n12 #CPUs=12 Partition=scavenge Nov 04 22:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_727(8157249) WEXITSTATUS 0 Nov 04 22:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_727(8157249) done Nov 04 22:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_847(8157390) WEXITSTATUS 0 Nov 04 22:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_847(8157390) done Nov 04 22:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_849(8157392) WEXITSTATUS 0 Nov 04 22:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_849(8157392) done Nov 04 22:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_921(8157575) WEXITSTATUS 0 Nov 04 22:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_921(8157575) done Nov 04 22:24:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_818(8157361) WEXITSTATUS 0 Nov 04 22:24:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_818(8157361) done Nov 04 22:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_992(8157646) WEXITSTATUS 0 Nov 04 22:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_992(8157646) done Nov 04 22:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_779(8157301) WEXITSTATUS 0 Nov 04 22:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_779(8157301) done Nov 04 22:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_901(8157555) WEXITSTATUS 0 Nov 04 22:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_901(8157555) done Nov 04 22:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_689(8157195) WEXITSTATUS 0 Nov 04 22:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_689(8157195) done Nov 04 22:24:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_908(8157562) WEXITSTATUS 0 Nov 04 22:24:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_908(8157562) done Nov 04 22:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_987(8157641) WEXITSTATUS 0 Nov 04 22:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_987(8157641) done Nov 04 22:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_919(8157573) WEXITSTATUS 0 Nov 04 22:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_919(8157573) done Nov 04 22:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_991(8157645) WEXITSTATUS 0 Nov 04 22:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_991(8157645) done Nov 04 22:24:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_923(8157577) WEXITSTATUS 0 Nov 04 22:24:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_923(8157577) done Nov 04 22:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2275(8152192) WEXITSTATUS 0 Nov 04 22:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2275(8152192) done Nov 04 22:24:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_914(8157568) WEXITSTATUS 0 Nov 04 22:24:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_914(8157568) done Nov 04 22:24:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_857(8157400) WEXITSTATUS 0 Nov 04 22:24:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_857(8157400) done Nov 04 22:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_941(8157595) WEXITSTATUS 0 Nov 04 22:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_941(8157595) done Nov 04 22:24:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_813(8157356) WEXITSTATUS 0 Nov 04 22:24:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_813(8157356) done Nov 04 22:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_798(8157320) WEXITSTATUS 0 Nov 04 22:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_798(8157320) done Nov 04 22:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_861(8157404) WEXITSTATUS 0 Nov 04 22:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_861(8157404) done Nov 04 22:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_905(8157559) WEXITSTATUS 0 Nov 04 22:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_905(8157559) done Nov 04 22:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_232(8156640) WEXITSTATUS 0 Nov 04 22:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_232(8156640) done Nov 04 22:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_848(8157391) WEXITSTATUS 0 Nov 04 22:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_848(8157391) done Nov 04 22:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_864(8157407) WEXITSTATUS 0 Nov 04 22:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_864(8157407) done Nov 04 22:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2126(8151802) WEXITSTATUS 0 Nov 04 22:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2126(8151802) done Nov 04 22:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_924(8157578) WEXITSTATUS 0 Nov 04 22:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_924(8157578) done Nov 04 22:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_969(8157623) WEXITSTATUS 0 Nov 04 22:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_969(8157623) done Nov 04 22:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_964(8157618) WEXITSTATUS 0 Nov 04 22:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_964(8157618) done Nov 04 22:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_955(8157609) WEXITSTATUS 0 Nov 04 22:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_955(8157609) done Nov 04 22:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4103(8148788) WEXITSTATUS 1 Nov 04 22:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4103(8148788) done Nov 04 22:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_692(8157198) WEXITSTATUS 0 Nov 04 22:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_692(8157198) done Nov 04 22:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156033 WEXITSTATUS 0 Nov 04 22:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156033 done Nov 04 22:24:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5172(8158526) WEXITSTATUS 1 Nov 04 22:24:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5172(8158526) done Nov 04 22:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_853(8157396) WEXITSTATUS 0 Nov 04 22:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_853(8157396) done Nov 04 22:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_933(8157587) WEXITSTATUS 0 Nov 04 22:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_933(8157587) done Nov 04 22:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_840(8157383) WEXITSTATUS 0 Nov 04 22:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_840(8157383) done Nov 04 22:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_855(8157398) WEXITSTATUS 0 Nov 04 22:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_855(8157398) done Nov 04 22:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_938(8157592) WEXITSTATUS 0 Nov 04 22:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_938(8157592) done Nov 04 22:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_780(8157302) WEXITSTATUS 0 Nov 04 22:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_780(8157302) done Nov 04 22:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_954(8157608) WEXITSTATUS 0 Nov 04 22:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_954(8157608) done Nov 04 22:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_925(8157579) WEXITSTATUS 0 Nov 04 22:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_925(8157579) done Nov 04 22:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_221(8156629) WEXITSTATUS 0 Nov 04 22:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_221(8156629) done Nov 04 22:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_786(8157308) WEXITSTATUS 0 Nov 04 22:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_786(8157308) done Nov 04 22:25:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158541 InitPrio=59549 usec=3937 Nov 04 22:25:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158542 InitPrio=59549 usec=4679 Nov 04 22:25:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158543 InitPrio=59549 usec=2886 Nov 04 22:25:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158544 InitPrio=59549 usec=2583 Nov 04 22:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_959(8157613) WEXITSTATUS 0 Nov 04 22:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_959(8157613) done Nov 04 22:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156031 WEXITSTATUS 0 Nov 04 22:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156031 done Nov 04 22:25:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158541 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 04 22:25:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158542 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 04 22:25:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158543 NodeList=r816u15n11 #CPUs=1 Partition=pi_anticevic Nov 04 22:25:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158544 NodeList=r816u15n11 #CPUs=1 Partition=pi_anticevic Nov 04 22:25:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5110(8158443) WEXITSTATUS 1 Nov 04 22:25:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5110(8158443) done Nov 04 22:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5109(8158442) WEXITSTATUS 1 Nov 04 22:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5109(8158442) done Nov 04 22:25:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157513 NodeList=r815u29n12 #CPUs=12 Partition=scavenge Nov 04 22:25:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157514 NodeList=r816u03n10 #CPUs=12 Partition=scavenge Nov 04 22:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_928(8157582) WEXITSTATUS 0 Nov 04 22:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_928(8157582) done Nov 04 22:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_809(8157352) WEXITSTATUS 0 Nov 04 22:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_809(8157352) done Nov 04 22:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_783(8157305) WEXITSTATUS 0 Nov 04 22:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_783(8157305) done Nov 04 22:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_858(8157401) WEXITSTATUS 0 Nov 04 22:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_858(8157401) done Nov 04 22:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_869(8157412) WEXITSTATUS 0 Nov 04 22:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_869(8157412) done Nov 04 22:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_841(8157384) WEXITSTATUS 0 Nov 04 22:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_841(8157384) done Nov 04 22:25:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_970(8157624) WEXITSTATUS 0 Nov 04 22:25:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_970(8157624) done Nov 04 22:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_994(8157648) WEXITSTATUS 0 Nov 04 22:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_994(8157648) done Nov 04 22:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_870(8157413) WEXITSTATUS 0 Nov 04 22:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_870(8157413) done Nov 04 22:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_782(8157304) WEXITSTATUS 0 Nov 04 22:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_782(8157304) done Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:25:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_856(8157399) WEXITSTATUS 0 Nov 04 22:25:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_856(8157399) done Nov 04 22:25:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:500 Nov 04 22:25:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4815(8157772) WEXITSTATUS 1 Nov 04 22:25:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4815(8157772) done Nov 04 22:25:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3542(8158545) in gpu on r4513u32n01 Nov 04 22:25:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3543(8158546) in gpu on r903u34n01 Nov 04 22:25:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1833(8158547) in day on r805u06n04 Nov 04 22:25:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1834(8158548) in day on r807u11n04 Nov 04 22:25:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1835(8158549) in day on r912u28n02 Nov 04 22:25:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157515 in scavenge on r816u15n03 Nov 04 22:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_906(8157560) WEXITSTATUS 0 Nov 04 22:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_906(8157560) done Nov 04 22:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_915(8157569) WEXITSTATUS 0 Nov 04 22:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_915(8157569) done Nov 04 22:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_790(8157312) WEXITSTATUS 0 Nov 04 22:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_790(8157312) done Nov 04 22:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_873(8157416) WEXITSTATUS 0 Nov 04 22:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_873(8157416) done Nov 04 22:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_907(8157561) WEXITSTATUS 0 Nov 04 22:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_907(8157561) done Nov 04 22:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_903(8157557) WEXITSTATUS 0 Nov 04 22:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_903(8157557) done Nov 04 22:25:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_785(8157307) WEXITSTATUS 0 Nov 04 22:25:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_785(8157307) done Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5187(8158550) in scavenge on r815u09n11 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5188(8158551) in scavenge on r815u29n09 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5189(8158552) in scavenge on r805u06n04 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5190(8158553) in scavenge on r909u01n03 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5191(8158554) in scavenge on r909u01n03 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5192(8158555) in scavenge on r904u04n01 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5193(8158556) in scavenge on r904u04n01 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5194(8158557) in scavenge on r904u04n01 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5195(8158558) in scavenge on r806u02n03 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5196(8158559) in scavenge on r806u02n04 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5197(8158560) in scavenge on r806u14n01 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5198(8158561) in scavenge on r806u14n01 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5199(8158562) in scavenge on r807u08n02 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5200(8158563) in scavenge on r807u08n02 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5201(8158564) in scavenge on r807u08n02 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5202(8158565) in scavenge on r807u08n03 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5203(8158566) in scavenge on r807u11n04 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5204(8158567) in scavenge on r807u11n04 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5205(8158568) in scavenge on r912u28n04 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5206(8158569) in scavenge on r912u36n02 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5207(8158570) in scavenge on r913u24n04 Nov 04 22:25:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5208(8158571) in scavenge on r806u31n01 Nov 04 22:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_867(8157410) WEXITSTATUS 0 Nov 04 22:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_867(8157410) done Nov 04 22:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_913(8157567) WEXITSTATUS 0 Nov 04 22:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_913(8157567) done Nov 04 22:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_973(8157627) WEXITSTATUS 0 Nov 04 22:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_973(8157627) done Nov 04 22:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4495(8155868) WEXITSTATUS 1 Nov 04 22:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4495(8155868) done Nov 04 22:25:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133246 WEXITSTATUS 0 Nov 04 22:25:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133246 done Nov 04 22:25:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158572 InitPrio=20786 usec=4162 Nov 04 22:25:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_846(8157389) WEXITSTATUS 0 Nov 04 22:25:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_846(8157389) done Nov 04 22:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_0(8158573) NodeList=r914u09n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_1(8158574) NodeList=r914u13n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:25:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_872(8157415) WEXITSTATUS 0 Nov 04 22:25:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_872(8157415) done Nov 04 22:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 20863: QOSMaxSubmitJobPerUserLimit Nov 04 22:25:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 22:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_918(8157572) WEXITSTATUS 0 Nov 04 22:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_918(8157572) done Nov 04 22:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_993(8157647) WEXITSTATUS 0 Nov 04 22:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_993(8157647) done Nov 04 22:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2243(8152035) WEXITSTATUS 0 Nov 04 22:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2243(8152035) done Nov 04 22:26:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_977(8157631) WEXITSTATUS 0 Nov 04 22:26:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_977(8157631) done Nov 04 22:26:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_862(8157405) WEXITSTATUS 0 Nov 04 22:26:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_862(8157405) done Nov 04 22:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_800(8157343) WEXITSTATUS 0 Nov 04 22:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_800(8157343) done Nov 04 22:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5041(8158337) WEXITSTATUS 1 Nov 04 22:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5041(8158337) done Nov 04 22:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4714(8156725) WEXITSTATUS 1 Nov 04 22:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4714(8156725) done Nov 04 22:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_494(8156959) WEXITSTATUS 0 Nov 04 22:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_494(8156959) done Nov 04 22:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5106(8158439) WEXITSTATUS 1 Nov 04 22:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5106(8158439) done Nov 04 22:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_995(8157649) WEXITSTATUS 0 Nov 04 22:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_995(8157649) done Nov 04 22:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_950(8157604) WEXITSTATUS 0 Nov 04 22:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_950(8157604) done Nov 04 22:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2293(8152210) WEXITSTATUS 0 Nov 04 22:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2293(8152210) done Nov 04 22:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4462(8155779) WEXITSTATUS 1 Nov 04 22:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4462(8155779) done Nov 04 22:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_863(8157406) WEXITSTATUS 0 Nov 04 22:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_863(8157406) done Nov 04 22:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_978(8157632) WEXITSTATUS 0 Nov 04 22:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_978(8157632) done Nov 04 22:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_882(8157425) WEXITSTATUS 0 Nov 04 22:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_882(8157425) done Nov 04 22:26:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158575 InitPrio=34421 usec=4226 Nov 04 22:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_851(8157394) WEXITSTATUS 0 Nov 04 22:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_851(8157394) done Nov 04 22:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_976(8157630) WEXITSTATUS 0 Nov 04 22:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_976(8157630) done Nov 04 22:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_886(8157429) WEXITSTATUS 0 Nov 04 22:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_886(8157429) done Nov 04 22:26:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1855(8124791) Nov 04 22:26:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4869(8157973) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:26:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4870(8157974) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_940(8157594) WEXITSTATUS 0 Nov 04 22:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_940(8157594) done Nov 04 22:26:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4870(8157974) Nov 04 22:26:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4869(8157973) Nov 04 22:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4528(8155929) WEXITSTATUS 1 Nov 04 22:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4528(8155929) done Nov 04 22:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_2(8158576) NodeList=r908u33n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_850(8157393) WEXITSTATUS 0 Nov 04 22:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_850(8157393) done Nov 04 22:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_45(8156168) WEXITSTATUS 0 Nov 04 22:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_45(8156168) done Nov 04 22:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_911(8157565) WEXITSTATUS 0 Nov 04 22:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_911(8157565) done Nov 04 22:26:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_902(8157556) WEXITSTATUS 0 Nov 04 22:26:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_902(8157556) done Nov 04 22:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156034 WEXITSTATUS 0 Nov 04 22:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156034 done Nov 04 22:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5137(8158484) WEXITSTATUS 1 Nov 04 22:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5137(8158484) done Nov 04 22:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3542(8158545) WEXITSTATUS 0 Nov 04 22:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3542(8158545) done Nov 04 22:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_962(8157616) WEXITSTATUS 0 Nov 04 22:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_962(8157616) done Nov 04 22:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_909(8157563) WEXITSTATUS 0 Nov 04 22:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_909(8157563) done Nov 04 22:26:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_966(8157620) WEXITSTATUS 0 Nov 04 22:26:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_966(8157620) done Nov 04 22:26:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_920(8157574) WEXITSTATUS 0 Nov 04 22:26:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_920(8157574) done Nov 04 22:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_965(8157619) WEXITSTATUS 0 Nov 04 22:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_965(8157619) done Nov 04 22:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_968(8157622) WEXITSTATUS 0 Nov 04 22:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_968(8157622) done Nov 04 22:27:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_953(8157607) WEXITSTATUS 0 Nov 04 22:27:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_953(8157607) done Nov 04 22:27:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_865(8157408) WEXITSTATUS 0 Nov 04 22:27:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_865(8157408) done Nov 04 22:27:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5082(8158406) WEXITSTATUS 1 Nov 04 22:27:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5082(8158406) done Nov 04 22:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_985(8157639) WEXITSTATUS 0 Nov 04 22:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_985(8157639) done Nov 04 22:27:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158575 NodeList=r807u35n01 #CPUs=8 Partition=bigmem Nov 04 22:27:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157516 NodeList=r816u23n06 #CPUs=12 Partition=scavenge Nov 04 22:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_972(8157626) WEXITSTATUS 0 Nov 04 22:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_972(8157626) done Nov 04 22:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_951(8157605) WEXITSTATUS 0 Nov 04 22:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_951(8157605) done Nov 04 22:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_974(8157628) WEXITSTATUS 0 Nov 04 22:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_974(8157628) done Nov 04 22:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156032 WEXITSTATUS 0 Nov 04 22:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156032 done Nov 04 22:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_961(8157615) WEXITSTATUS 0 Nov 04 22:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_961(8157615) done Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:27:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:27:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4879(8157990) has been requeued to reclaim resources for JobId=8158572_3(8158572) Nov 04 22:27:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4880(8157991) has been requeued to reclaim resources for JobId=8158572_3(8158572) Nov 04 22:27:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4879(8157990) Nov 04 22:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_986(8157640) WEXITSTATUS 0 Nov 04 22:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_986(8157640) done Nov 04 22:27:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4880(8157991) Nov 04 22:27:31 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_3(8158577) NodeList=r908u35n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:27:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5150(8158497) WEXITSTATUS 1 Nov 04 22:27:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5150(8158497) done Nov 04 22:27:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_960(8157614) WEXITSTATUS 0 Nov 04 22:27:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_960(8157614) done Nov 04 22:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2129(8151805) WEXITSTATUS 0 Nov 04 22:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2129(8151805) done Nov 04 22:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_879(8157422) WEXITSTATUS 0 Nov 04 22:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_879(8157422) done Nov 04 22:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_971(8157625) WEXITSTATUS 0 Nov 04 22:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_971(8157625) done Nov 04 22:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_967(8157621) WEXITSTATUS 0 Nov 04 22:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_967(8157621) done Nov 04 22:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_871(8157414) WEXITSTATUS 0 Nov 04 22:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_871(8157414) done Nov 04 22:28:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_198(7572366) Nov 04 22:28:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_387(7572555) Nov 04 22:28:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_390(7572558) Nov 04 22:28:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4881(8157992) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:28:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4882(8157993) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:28:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4881(8157992) Nov 04 22:28:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4882(8157993) Nov 04 22:28:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_4(8158578) NodeList=r908u33n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1166(7572133) WEXITSTATUS 0 Nov 04 22:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1166(7572133) done Nov 04 22:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_957(8157611) WEXITSTATUS 0 Nov 04 22:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_957(8157611) done Nov 04 22:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4863(8157962) WEXITSTATUS 1 Nov 04 22:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4863(8157962) done Nov 04 22:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_881(8157424) WEXITSTATUS 0 Nov 04 22:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_881(8157424) done Nov 04 22:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_963(8157617) WEXITSTATUS 0 Nov 04 22:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_963(8157617) done Nov 04 22:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_975(8157629) WEXITSTATUS 0 Nov 04 22:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_975(8157629) done Nov 04 22:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_884(8157427) WEXITSTATUS 0 Nov 04 22:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_884(8157427) done Nov 04 22:28:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_880(8157423) WEXITSTATUS 0 Nov 04 22:28:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_880(8157423) done Nov 04 22:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_982(8157636) WEXITSTATUS 0 Nov 04 22:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_982(8157636) done Nov 04 22:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_956(8157610) WEXITSTATUS 0 Nov 04 22:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_956(8157610) done Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_984(8157638) WEXITSTATUS 0 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_984(8157638) done Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4967(8158229) WEXITSTATUS 1 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4967(8158229) done Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_5(8158579) in day on r909u01n01 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_6(8158580) in day on r918u05n03 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_7(8158581) in day on r912u28n03 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_8(8158582) in day on r913u28n01 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_9(8158583) in day on r806u02n01 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_10(8158584) in day on r806u09n01 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_11(8158585) in day on r806u11n01 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_12(8158586) in day on r807u05n01 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_13(8158587) in day on r807u05n02 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_14(8158588) in day on r807u08n04 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_15(8158589) in day on r805u04n04 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_16(8158590) in day on r805u06n02 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_17(8158591) in day on r805u11n04 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_18(8158592) in day on r805u14n01 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_19(8158593) in day on r805u25n04 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_20(8158594) in day on r805u27n02 Nov 04 22:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3544(8158595) in gpu on r4513u32n01 Nov 04 22:28:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 22:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_980(8157634) WEXITSTATUS 0 Nov 04 22:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_980(8157634) done Nov 04 22:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_523(8157008) WEXITSTATUS 0 Nov 04 22:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_523(8157008) done Nov 04 22:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_883(8157426) WEXITSTATUS 0 Nov 04 22:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_883(8157426) done Nov 04 22:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2038(8151598) WEXITSTATUS 0 Nov 04 22:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2038(8151598) done Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5209(8158596) in scavenge on r801u06n02 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5210(8158597) in scavenge on r802u02n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5211(8158598) in scavenge on r903u01n03 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5212(8158599) in scavenge on r907u34n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5213(8158600) in scavenge on r914u05n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5214(8158601) in scavenge on r805u04n03 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5215(8158602) in scavenge on r805u09n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5216(8158603) in scavenge on r806u04n04 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5217(8158604) in scavenge on r806u14n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5218(8158605) in scavenge on r909u01n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5219(8158606) in scavenge on r909u13n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5220(8158607) in scavenge on r909u13n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5221(8158608) in scavenge on r918u05n03 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5222(8158609) in scavenge on r919u32n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5223(8158610) in scavenge on r807u03n04 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5224(8158611) in scavenge on r807u05n01 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5225(8158612) in scavenge on r807u05n04 Nov 04 22:28:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5226(8158613) in scavenge on r816u35n06 Nov 04 22:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_885(8157428) WEXITSTATUS 0 Nov 04 22:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_885(8157428) done Nov 04 22:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2093(8151769) WEXITSTATUS 0 Nov 04 22:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2093(8151769) done Nov 04 22:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_981(8157635) WEXITSTATUS 0 Nov 04 22:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_981(8157635) done Nov 04 22:29:02 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4911(8158074) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:29:02 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5077(8158389) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:29:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4911(8158074) Nov 04 22:29:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5077(8158389) Nov 04 22:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_21(8158614) NodeList=r908u39n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1836(8158615) in scavenge on r911u05n01 Nov 04 22:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1837(8158616) in scavenge on r911u05n01 Nov 04 22:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1838(8158617) in scavenge on r911u05n01 Nov 04 22:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1839(8158618) in scavenge on r911u05n03 Nov 04 22:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1840(8158619) in scavenge on r806u31n01 Nov 04 22:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1841(8158620) in scavenge on r806u31n01 Nov 04 22:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2187(8151979) WEXITSTATUS 0 Nov 04 22:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2187(8151979) done Nov 04 22:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_6(8156129) WEXITSTATUS 0 Nov 04 22:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_6(8156129) done Nov 04 22:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_887(8157430) WEXITSTATUS 0 Nov 04 22:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_887(8157430) done Nov 04 22:29:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5164(8158518) WEXITSTATUS 1 Nov 04 22:29:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5164(8158518) done Nov 04 22:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_983(8157637) WEXITSTATUS 0 Nov 04 22:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_983(8157637) done Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:29:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:29:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:29:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_868(8157411) WEXITSTATUS 0 Nov 04 22:29:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_868(8157411) done Nov 04 22:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_952(8157606) WEXITSTATUS 0 Nov 04 22:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_952(8157606) done Nov 04 22:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4874(8157978) WEXITSTATUS 1 Nov 04 22:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4874(8157978) done Nov 04 22:29:58 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4867(8157971) has been requeued to reclaim resources for JobId=8158572_22(8158572) Nov 04 22:29:58 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4983(8158256) has been requeued to reclaim resources for JobId=8158572_22(8158572) Nov 04 22:29:59 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4867(8157971) Nov 04 22:30:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4983(8158256) Nov 04 22:30:00 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_22(8158621) NodeList=r908u39n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135382 WEXITSTATUS 0 Nov 04 22:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135382 done Nov 04 22:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5211(8158598) WEXITSTATUS 1 Nov 04 22:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5211(8158598) done Nov 04 22:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_958(8157612) WEXITSTATUS 0 Nov 04 22:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_958(8157612) done Nov 04 22:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_28(8135727) WEXITSTATUS 0 Nov 04 22:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_28(8135727) done Nov 04 22:30:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4884(8157995) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:30:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5014(8158305) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:30:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5014(8158305) Nov 04 22:30:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4884(8157995) Nov 04 22:30:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_23(8158622) NodeList=r908u33n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4799(8157695) WEXITSTATUS 1 Nov 04 22:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4799(8157695) done Nov 04 22:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2276(8152193) WEXITSTATUS 0 Nov 04 22:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2276(8152193) done Nov 04 22:30:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158623 InitPrio=30493 usec=4355 Nov 04 22:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2140(8151816) WEXITSTATUS 0 Nov 04 22:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2140(8151816) done Nov 04 22:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2188(8151980) WEXITSTATUS 0 Nov 04 22:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2188(8151980) done Nov 04 22:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3531(8158404) WEXITSTATUS 0 Nov 04 22:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3531(8158404) done Nov 04 22:31:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4896(8158029) WEXITSTATUS 1 Nov 04 22:31:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4896(8158029) done Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:31:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2236(8152028) WEXITSTATUS 0 Nov 04 22:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2236(8152028) done Nov 04 22:31:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4861(8157960) has been requeued to reclaim resources for JobId=8158572_24(8158572) Nov 04 22:31:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5100(8158433) has been requeued to reclaim resources for JobId=8158572_24(8158572) Nov 04 22:31:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5100(8158433) Nov 04 22:31:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4861(8157960) Nov 04 22:31:31 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158572_24(8158624) NodeList=r908u35n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 04 22:31:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_0(8149881) Nov 04 22:31:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_1(8149882) Nov 04 22:31:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_2(8149883) Nov 04 22:31:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4067(8147920) WEXITSTATUS 1 Nov 04 22:31:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4067(8147920) done Nov 04 22:31:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156036 WEXITSTATUS 0 Nov 04 22:31:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156036 done Nov 04 22:31:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 04 22:31:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n03 Nov 04 22:31:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_25(8158625) in day on r912u28n02 Nov 04 22:31:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3545(8158626) in gpu on r808u11n01 Nov 04 22:31:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1842(8158627) in day on r914u01n01 Nov 04 22:31:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1843(8158628) in day on r815u03n07 Nov 04 22:31:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1844(8158629) in day on r815u03n07 Nov 04 22:31:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 22:31:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157517 in scavenge on r816u35n10 Nov 04 22:31:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157518 in scavenge on r815u03n07 Nov 04 22:31:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157519 in scavenge on r815u03n07 Nov 04 22:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3540(8158505) WEXITSTATUS 0 Nov 04 22:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3540(8158505) done Nov 04 22:31:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4564(8156021) WEXITSTATUS 1 Nov 04 22:31:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4564(8156021) done Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5227(8158630) in scavenge on r907u01n02 Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5228(8158631) in scavenge on r802u09n02 Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5229(8158632) in scavenge on r903u01n03 Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5230(8158633) in scavenge on r903u04n01 Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5231(8158634) in scavenge on r805u09n03 Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5232(8158635) in scavenge on r807u11n04 Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5233(8158636) in scavenge on r918u01n03 Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5234(8158637) in scavenge on r912u28n02 Nov 04 22:31:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5235(8158638) in scavenge on r912u32n03 Nov 04 22:32:00 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 22:32:02 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4728(8156857) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:32:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 04 22:32:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 04 22:32:04 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 04 22:32:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4728(8156857) Nov 04 22:32:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158623 NodeList=r816u35n07,r907u32n01 #CPUs=8 Partition=gpu Nov 04 22:32:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_550(8157035) WEXITSTATUS 0 Nov 04 22:32:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_550(8157035) done Nov 04 22:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158623 WEXITSTATUS 1 Nov 04 22:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158623 done Nov 04 22:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156035 WEXITSTATUS 0 Nov 04 22:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156035 done Nov 04 22:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4286(8154052) WEXITSTATUS 1 Nov 04 22:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4286(8154052) done Nov 04 22:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5151(8158498) WEXITSTATUS 1 Nov 04 22:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5151(8158498) done Nov 04 22:32:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_632(8157138) WEXITSTATUS 0 Nov 04 22:32:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_632(8157138) done Nov 04 22:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5153(8158500) WEXITSTATUS 1 Nov 04 22:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5153(8158500) done Nov 04 22:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4614(8156193) WEXITSTATUS 1 Nov 04 22:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4614(8156193) done Nov 04 22:33:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1860(8124817) Nov 04 22:33:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5144(8158491) WEXITSTATUS 1 Nov 04 22:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5144(8158491) done Nov 04 22:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4551(8155960) WEXITSTATUS 1 Nov 04 22:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4551(8155960) done Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:33:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:33:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158639 InitPrio=48651 usec=7683 Nov 04 22:33:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5206(8158569) WEXITSTATUS 1 Nov 04 22:33:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5206(8158569) done Nov 04 22:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2333(8152250) WEXITSTATUS 0 Nov 04 22:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2333(8152250) done Nov 04 22:34:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5160(8158514) WEXITSTATUS 1 Nov 04 22:34:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5160(8158514) done Nov 04 22:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158639 NodeList=r805u30n01 #CPUs=8 Partition=gpu_devel Nov 04 22:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5194(8158557) WEXITSTATUS 1 Nov 04 22:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5194(8158557) done Nov 04 22:34:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4935(8158144) WEXITSTATUS 1 Nov 04 22:34:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4935(8158144) done Nov 04 22:34:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4972(8158234) WEXITSTATUS 1 Nov 04 22:34:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4972(8158234) done Nov 04 22:34:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2097(8151773) WEXITSTATUS 0 Nov 04 22:34:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2097(8151773) done Nov 04 22:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3544(8158595) WEXITSTATUS 0 Nov 04 22:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3544(8158595) done Nov 04 22:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4096(8148576) WEXITSTATUS 1 Nov 04 22:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4096(8148576) done Nov 04 22:35:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_26(8158640) in day on r806u06n03 Nov 04 22:35:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3546(8158641) in gpu on r903u40n01 Nov 04 22:35:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3547(8158642) in gpu on r4513u26n01 Nov 04 22:35:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157520 in scavenge on r815u09n05 Nov 04 22:35:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8143160 Nov 04 22:35:03 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_3(8150054) Nov 04 22:35:03 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_4(8150055) Nov 04 22:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1949(8151387) WEXITSTATUS 0 Nov 04 22:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1949(8151387) done Nov 04 22:35:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_747(8157269) WEXITSTATUS 0 Nov 04 22:35:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_747(8157269) done Nov 04 22:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5135(8158475) WEXITSTATUS 1 Nov 04 22:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5135(8158475) done Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5236(8158643) in scavenge on r907u05n02 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5237(8158644) in scavenge on r903u01n01 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5238(8158645) in scavenge on r915u35n04 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5239(8158646) in scavenge on r917u07n04 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5240(8158647) in scavenge on r4513u32n01 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5241(8158648) in scavenge on r816u35n07 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5242(8158649) in scavenge on r904u04n01 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5243(8158650) in scavenge on r918u36n01 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5244(8158651) in scavenge on r912u36n02 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5245(8158652) in scavenge on r913u01n02 Nov 04 22:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5246(8158653) in scavenge on r913u13n04 Nov 04 22:35:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2367(8152411) WEXITSTATUS 0 Nov 04 22:35:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2367(8152411) done Nov 04 22:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2204(8151996) WEXITSTATUS 0 Nov 04 22:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2204(8151996) done Nov 04 22:35:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_290(7572458) Nov 04 22:35:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1123(7573292) Nov 04 22:35:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_485(8114752) Nov 04 22:35:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1865(8124832) Nov 04 22:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156037 WEXITSTATUS 0 Nov 04 22:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156037 done Nov 04 22:35:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5187(8158550) WEXITSTATUS 1 Nov 04 22:35:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5187(8158550) done Nov 04 22:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4929(8158116) WEXITSTATUS 1 Nov 04 22:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4929(8158116) done Nov 04 22:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_671(8157177) WEXITSTATUS 0 Nov 04 22:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_671(8157177) done Nov 04 22:35:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2134(8151810) WEXITSTATUS 0 Nov 04 22:35:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2134(8151810) done Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:35:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4895(8158028) WEXITSTATUS 1 Nov 04 22:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4895(8158028) done Nov 04 22:35:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1840(8151166) WEXITSTATUS 0 Nov 04 22:35:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1840(8151166) done Nov 04 22:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2485(8127540) WEXITSTATUS 1 Nov 04 22:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2485(8127540) done Nov 04 22:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4322(8154205) WEXITSTATUS 1 Nov 04 22:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4322(8154205) done Nov 04 22:36:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157521 NodeList=r816u29n08 #CPUs=12 Partition=scavenge Nov 04 22:36:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157522 NodeList=r816u29n08 #CPUs=12 Partition=scavenge Nov 04 22:36:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157523 NodeList=r816u29n03 #CPUs=12 Partition=scavenge Nov 04 22:36:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157524 NodeList=r816u29n06 #CPUs=12 Partition=scavenge Nov 04 22:36:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5236(8158643) WEXITSTATUS 1 Nov 04 22:36:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5236(8158643) done Nov 04 22:36:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4592(8156113) WEXITSTATUS 1 Nov 04 22:36:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4592(8156113) done Nov 04 22:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5229(8158632) WEXITSTATUS 1 Nov 04 22:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5229(8158632) done Nov 04 22:36:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158248 WEXITSTATUS 0 Nov 04 22:36:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158248 done Nov 04 22:36:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158654 InitPrio=10837 usec=4897 Nov 04 22:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3460(8157444) WEXITSTATUS 0 Nov 04 22:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3460(8157444) done Nov 04 22:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1829(8151155) WEXITSTATUS 0 Nov 04 22:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1829(8151155) done Nov 04 22:36:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158655 InitPrio=10837 usec=3934 Nov 04 22:37:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158656 InitPrio=10837 usec=5073 Nov 04 22:37:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158657 InitPrio=10837 usec=4489 Nov 04 22:37:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158658 InitPrio=20786 usec=3828 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:37:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3502(8158069) WEXITSTATUS 0 Nov 04 22:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3502(8158069) done Nov 04 22:38:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_691(7572860) Nov 04 22:38:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1083(7573252) Nov 04 22:38:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_5(8150376) Nov 04 22:38:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_7(8150378) Nov 04 22:38:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2018(8151578) WEXITSTATUS 0 Nov 04 22:38:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2018(8151578) done Nov 04 22:38:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157525 NodeList=r815u23n07 #CPUs=12 Partition=scavenge Nov 04 22:38:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157526 NodeList=r816u23n12 #CPUs=12 Partition=scavenge Nov 04 22:38:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157527 NodeList=r816u29n11 #CPUs=12 Partition=scavenge Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158654 in pi_ohern on r917u13n02 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158655 in pi_ohern on r917u13n02 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158656 in pi_ohern on r917u13n02 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158657 in pi_ohern on r917u13n02 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_27(8158659) in day on r913u24n02 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_28(8158660) in day on r815u23n04 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_29(8158661) in day on r815u03n04 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_30(8158662) in day on r815u09n06 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_31(8158663) in day on r815u09n06 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_32(8158664) in day on r816u03n02 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_33(8158665) in day on r816u03n07 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_34(8158666) in day on r816u03n07 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_35(8158667) in day on r816u15n02 Nov 04 22:38:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_36(8158668) in day on r816u15n04 Nov 04 22:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3548(8158669) in gpu on r808u08n01 Nov 04 22:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3549(8158670) in gpu on r808u14n01 Nov 04 22:38:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_335(7571300) WEXITSTATUS 0 Nov 04 22:38:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_335(7571300) done Nov 04 22:38:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1921(8151359) WEXITSTATUS 0 Nov 04 22:38:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1921(8151359) done Nov 04 22:38:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n01 Nov 04 22:38:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157528 in scavenge on r815u03n06 Nov 04 22:38:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4840(8157848) WEXITSTATUS 1 Nov 04 22:38:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4840(8157848) done Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5147(8158494) WEXITSTATUS 1 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5147(8158494) done Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5247(8158671) in scavenge on r907u05n02 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5248(8158672) in scavenge on r801u14n03 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5249(8158673) in scavenge on r802u02n03 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5250(8158674) in scavenge on r903u01n03 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5251(8158675) in scavenge on r815u09n11 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5252(8158676) in scavenge on r806u02n04 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5253(8158677) in scavenge on r909u13n04 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5254(8158678) in scavenge on r913u24n04 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5255(8158679) in scavenge on r815u03n05 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5256(8158680) in scavenge on r815u09n07 Nov 04 22:38:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5257(8158681) in scavenge on r908u24n02 Nov 04 22:38:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4931(8158118) WEXITSTATUS 1 Nov 04 22:38:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4931(8158118) done Nov 04 22:38:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3548(8158669) WEXITSTATUS 0 Nov 04 22:38:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3548(8158669) done Nov 04 22:38:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2314(8152231) WEXITSTATUS 0 Nov 04 22:38:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2314(8152231) done Nov 04 22:38:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1845(8158682) in scavenge on r815u23n07 Nov 04 22:38:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1846(8158683) in scavenge on r815u23n07 Nov 04 22:38:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1847(8158684) in scavenge on r815u23n07 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:39:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4919(8158082) WEXITSTATUS 1 Nov 04 22:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4919(8158082) done Nov 04 22:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4266(8153748) WEXITSTATUS 1 Nov 04 22:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4266(8153748) done Nov 04 22:39:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5200(8158563) WEXITSTATUS 1 Nov 04 22:39:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5200(8158563) done Nov 04 22:39:37 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2132(8151808) WEXITSTATUS 0 Nov 04 22:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2132(8151808) done Nov 04 22:39:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4573(8156082) WEXITSTATUS 1 Nov 04 22:39:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4573(8156082) done Nov 04 22:39:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4767(8157449) WEXITSTATUS 1 Nov 04 22:39:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4767(8157449) done Nov 04 22:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_32(8135731) WEXITSTATUS 0 Nov 04 22:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_32(8135731) done Nov 04 22:39:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4393(8155323) WEXITSTATUS 1 Nov 04 22:39:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4393(8155323) done Nov 04 22:39:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3527(8158268) WEXITSTATUS 0 Nov 04 22:39:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3527(8158268) done Nov 04 22:39:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5175(8158529) WEXITSTATUS 1 Nov 04 22:39:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5175(8158529) done Nov 04 22:39:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158423 uid 14201 Nov 04 22:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1926(8151364) WEXITSTATUS 0 Nov 04 22:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1926(8151364) done Nov 04 22:40:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5237(8158644) WEXITSTATUS 1 Nov 04 22:40:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5237(8158644) done Nov 04 22:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4937(8158146) WEXITSTATUS 1 Nov 04 22:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4937(8158146) done Nov 04 22:40:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_5(8157553) WEXITSTATUS 0 Nov 04 22:40:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_5(8157553) done Nov 04 22:40:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4437(8155678) WEXITSTATUS 1 Nov 04 22:40:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4437(8155678) done Nov 04 22:40:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3549(8158670) WEXITSTATUS 0 Nov 04 22:40:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3549(8158670) done Nov 04 22:40:36 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_397(7572565) Nov 04 22:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4798(8157694) WEXITSTATUS 1 Nov 04 22:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4798(8157694) done Nov 04 22:40:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156038 WEXITSTATUS 0 Nov 04 22:40:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156038 done Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:41:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:41:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_8(8150501) Nov 04 22:41:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_9(8150502) Nov 04 22:41:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157529 NodeList=r816u23n07 #CPUs=12 Partition=scavenge Nov 04 22:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1228(8135030) WEXITSTATUS 0 Nov 04 22:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1228(8135030) done Nov 04 22:41:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:25 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:25 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:26 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:41:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_37(8158685) in day on r912u32n01 Nov 04 22:41:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_38(8158686) in day on r805u06n04 Nov 04 22:41:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_39(8158687) in day on r805u14n02 Nov 04 22:41:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_40(8158688) in day on r815u15n02 Nov 04 22:41:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3550(8158689) in gpu on r808u08n01 Nov 04 22:41:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3551(8158690) in gpu on r808u08n01 Nov 04 22:41:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3552(8158691) in gpu on r808u14n01 Nov 04 22:41:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_986(7571952) WEXITSTATUS 0 Nov 04 22:41:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_986(7571952) done Nov 04 22:41:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1071(7572037) WEXITSTATUS 0 Nov 04 22:41:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1071(7572037) done Nov 04 22:41:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_635(7572804) WEXITSTATUS 0 Nov 04 22:41:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_635(7572804) done Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5258(8158692) in scavenge on r908u28n02 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5259(8158693) in scavenge on r907u05n01 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5260(8158694) in scavenge on r903u01n01 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5261(8158695) in scavenge on r802u02n02 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5262(8158696) in scavenge on r802u32n01 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5263(8158697) in scavenge on r805u25n02 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5264(8158698) in scavenge on r914u05n04 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5265(8158699) in scavenge on r914u05n04 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5266(8158700) in scavenge on r914u05n04 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5267(8158701) in scavenge on r806u04n04 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5268(8158702) in scavenge on r806u06n01 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5269(8158703) in scavenge on r912u28n04 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5270(8158704) in scavenge on r912u36n01 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5271(8158705) in scavenge on r912u36n03 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5272(8158706) in scavenge on r918u36n02 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5273(8158707) in scavenge on r919u09n03 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5274(8158708) in scavenge on r919u32n01 Nov 04 22:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5275(8158709) in scavenge on r815u03n03 Nov 04 22:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_728(8157250) WEXITSTATUS 0 Nov 04 22:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_728(8157250) done Nov 04 22:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2303(8152220) WEXITSTATUS 0 Nov 04 22:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2303(8152220) done Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:42:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:42:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4856(8157941) WEXITSTATUS 1 Nov 04 22:42:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4856(8157941) done Nov 04 22:42:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5267(8158701) WEXITSTATUS 1 Nov 04 22:42:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5267(8158701) done Nov 04 22:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5048(8158344) WEXITSTATUS 1 Nov 04 22:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5048(8158344) done Nov 04 22:43:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5216(8158603) WEXITSTATUS 1 Nov 04 22:43:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5216(8158603) done Nov 04 22:43:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_412(7572580) Nov 04 22:43:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_417(7572585) Nov 04 22:43:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157530 NodeList=r816u03n10 #CPUs=12 Partition=scavenge Nov 04 22:43:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2393(8152437) WEXITSTATUS 0 Nov 04 22:43:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2393(8152437) done Nov 04 22:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4650(8156254) WEXITSTATUS 1 Nov 04 22:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4650(8156254) done Nov 04 22:43:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158710 InitPrio=37978 usec=7525 Nov 04 22:43:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158710 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 22:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_979(8157633) WEXITSTATUS 0 Nov 04 22:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_979(8157633) done Nov 04 22:43:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5240(8158647) WEXITSTATUS 1 Nov 04 22:43:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5240(8158647) done Nov 04 22:43:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4912(8158075) WEXITSTATUS 1 Nov 04 22:43:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4912(8158075) done Nov 04 22:44:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_10(8150714) Nov 04 22:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3538(8158477) WEXITSTATUS 0 Nov 04 22:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3538(8158477) done Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:44:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:44:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3552(8158691) WEXITSTATUS 0 Nov 04 22:44:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3552(8158691) done Nov 04 22:44:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5242(8158649) WEXITSTATUS 1 Nov 04 22:44:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5242(8158649) done Nov 04 22:44:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_41(8158711) in day on r805u25n01 Nov 04 22:44:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_42(8158712) in day on r806u04n04 Nov 04 22:44:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_43(8158713) in day on r807u05n03 Nov 04 22:44:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_44(8158714) in day on r909u07n03 Nov 04 22:44:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_45(8158715) in day on r912u32n01 Nov 04 22:44:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3553(8158716) in gpu on r808u14n01 Nov 04 22:44:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3554(8158717) in gpu on r808u17n01 Nov 04 22:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_76(7572243) WEXITSTATUS 0 Nov 04 22:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_76(7572243) done Nov 04 22:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_95(7572262) WEXITSTATUS 0 Nov 04 22:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_95(7572262) done Nov 04 22:44:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n01 Nov 04 22:45:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8138141 Nov 04 22:45:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5276(8158718) in scavenge on r802u06n01 Nov 04 22:45:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5277(8158719) in scavenge on r917u07n04 Nov 04 22:45:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5278(8158720) in scavenge on r4513u32n01 Nov 04 22:45:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5279(8158721) in scavenge on r805u09n04 Nov 04 22:45:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5280(8158722) in scavenge on r909u07n03 Nov 04 22:45:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5281(8158723) in scavenge on r919u09n02 Nov 04 22:45:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2113(8151789) WEXITSTATUS 0 Nov 04 22:45:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2113(8151789) done Nov 04 22:45:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4875(8157979) WEXITSTATUS 1 Nov 04 22:45:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4875(8157979) done Nov 04 22:45:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3553(8158716) WEXITSTATUS 0 Nov 04 22:45:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3553(8158716) done Nov 04 22:45:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3547(8158642) WEXITSTATUS 0 Nov 04 22:45:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3547(8158642) done Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:45:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5281(8158723) WEXITSTATUS 1 Nov 04 22:45:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5281(8158723) done Nov 04 22:45:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5045(8158341) WEXITSTATUS 1 Nov 04 22:45:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5045(8158341) done Nov 04 22:46:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:46:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5034(8158330) WEXITSTATUS 1 Nov 04 22:46:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5034(8158330) done Nov 04 22:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156040 WEXITSTATUS 0 Nov 04 22:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156040 done Nov 04 22:46:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4583(8156096) WEXITSTATUS 1 Nov 04 22:46:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4583(8156096) done Nov 04 22:46:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3432(8135273) WEXITSTATUS 1 Nov 04 22:46:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3432(8135273) done Nov 04 22:46:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4916(8158079) WEXITSTATUS 1 Nov 04 22:46:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4916(8158079) done Nov 04 22:46:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146706 Nov 04 22:46:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4994(8158273) WEXITSTATUS 1 Nov 04 22:46:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4994(8158273) done Nov 04 22:46:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2346(8152390) WEXITSTATUS 0 Nov 04 22:46:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2346(8152390) done Nov 04 22:47:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_11(8150925) Nov 04 22:47:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_12(8150926) Nov 04 22:47:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_38(8156161) WEXITSTATUS 0 Nov 04 22:47:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_38(8156161) done Nov 04 22:47:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158724 InitPrio=34228 usec=3824 Nov 04 22:47:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158724 NodeList=r801u06n01 #CPUs=4 Partition=education Nov 04 22:47:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157531 NodeList=r815u15n11 #CPUs=12 Partition=scavenge Nov 04 22:47:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5266(8158700) WEXITSTATUS 1 Nov 04 22:47:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5266(8158700) done Nov 04 22:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158724 WEXITSTATUS 64 Nov 04 22:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158724 done Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:47:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:47:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4976(8158238) WEXITSTATUS 1 Nov 04 22:47:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4976(8158238) done Nov 04 22:47:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156041 WEXITSTATUS 0 Nov 04 22:47:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156041 done Nov 04 22:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4788(8157663) WEXITSTATUS 1 Nov 04 22:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4788(8157663) done Nov 04 22:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_942(8157596) WEXITSTATUS 0 Nov 04 22:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_942(8157596) done Nov 04 22:48:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_46(8158725) in day on r805u11n02 Nov 04 22:48:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_47(8158726) in day on r807u08n01 Nov 04 22:48:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_48(8158727) in day on r918u01n02 Nov 04 22:48:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158572_49(8158572) in day on r918u05n03 Nov 04 22:48:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3555(8158728) in gpu on r808u14n01 Nov 04 22:48:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3556(8158729) in gpu on r4513u26n01 Nov 04 22:48:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157532 in scavenge on r816u29n07 Nov 04 22:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4964(8158214) WEXITSTATUS 1 Nov 04 22:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4964(8158214) done Nov 04 22:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144198 in scavenge on r807u01n04,r907u40n01 Nov 04 22:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2278(8152195) WEXITSTATUS 0 Nov 04 22:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2278(8152195) done Nov 04 22:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4773(8157455) WEXITSTATUS 1 Nov 04 22:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4773(8157455) done Nov 04 22:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2148(8151824) WEXITSTATUS 0 Nov 04 22:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2148(8151824) done Nov 04 22:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3556(8158729) WEXITSTATUS 0 Nov 04 22:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3556(8158729) done Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5282(8158730) in scavenge on r801u11n04 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5283(8158731) in scavenge on r915u35n02 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5284(8158732) in scavenge on r808u08n01 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5285(8158733) in scavenge on r805u02n04 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5286(8158734) in scavenge on r909u13n03 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5287(8158735) in scavenge on r904u04n01 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5288(8158736) in scavenge on r912u36n02 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5289(8158737) in scavenge on r914u05n04 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5290(8158738) in scavenge on r807u03n04 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5291(8158739) in scavenge on r807u08n02 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5292(8158740) in scavenge on r807u11n02 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5293(8158741) in scavenge on r918u01n02 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5294(8158742) in scavenge on r918u32n02 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5295(8158743) in scavenge on r919u05n01 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5296(8158744) in scavenge on r919u05n04 Nov 04 22:48:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5297(8158745) in scavenge on r919u09n02 Nov 04 22:48:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_0(8158746) in scavenge on r802u25n04 Nov 04 22:48:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_1(8158747) in scavenge on r903u01n02 Nov 04 22:48:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_2(8158748) in scavenge on r912u01n02 Nov 04 22:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4582(8156095) WEXITSTATUS 1 Nov 04 22:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4582(8156095) done Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3491(8157952) WEXITSTATUS 0 Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3491(8157952) done Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_979(7573148) Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1884(8124951) Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_866(8157409) WEXITSTATUS 0 Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2114 TRES cpu grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2114 TRES mem grp_used_tres_run_secs underflow, tried to remove 28549120 seconds when only 28303360 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2114 TRES cpu grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2114 TRES node grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2114 TRES billing grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2113 TRES cpu grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2113 TRES mem grp_used_tres_run_secs underflow, tried to remove 28549120 seconds when only 28303360 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2113 TRES node grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2113 TRES billing grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156351_866(8157409) done Nov 04 22:48:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2114 TRES mem grp_used_tres_run_secs underflow, tried to remove 28549120 seconds when only 28303360 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2114 TRES node grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2114 TRES billing grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2113 TRES cpu grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2113 TRES mem grp_used_tres_run_secs underflow, tried to remove 28549120 seconds when only 28303360 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2113 TRES node grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8157409: assoc 2113 TRES billing grp_used_tres_run_secs underflow, tried to remove 6970 seconds when only 6910 remained. Nov 04 22:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3521(8158226) WEXITSTATUS 0 Nov 04 22:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3521(8158226) done Nov 04 22:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4915(8158078) WEXITSTATUS 1 Nov 04 22:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4915(8158078) done Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:49:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2282(8152199) WEXITSTATUS 0 Nov 04 22:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2282(8152199) done Nov 04 22:49:15 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157533 NodeList=r816u35n10 #CPUs=12 Partition=scavenge Nov 04 22:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2337(8152254) WEXITSTATUS 0 Nov 04 22:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2337(8152254) done Nov 04 22:49:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3451(8157208) WEXITSTATUS 0 Nov 04 22:49:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3451(8157208) done Nov 04 22:49:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158394 uid 21916 Nov 04 22:49:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158749 InitPrio=47155 usec=4550 Nov 04 22:49:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156039 WEXITSTATUS 0 Nov 04 22:49:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156039 done Nov 04 22:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3518(8158223) WEXITSTATUS 0 Nov 04 22:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3518(8158223) done Nov 04 22:50:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2320(8152237) WEXITSTATUS 0 Nov 04 22:50:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2320(8152237) done Nov 04 22:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4958(8158208) WEXITSTATUS 1 Nov 04 22:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4958(8158208) done Nov 04 22:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4330(8154394) WEXITSTATUS 1 Nov 04 22:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4330(8154394) done Nov 04 22:50:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156043 WEXITSTATUS 0 Nov 04 22:50:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156043 done Nov 04 22:50:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146923 Nov 04 22:50:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157534 NodeList=r815u23n08 #CPUs=12 Partition=scavenge Nov 04 22:50:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158750 InitPrio=36143 usec=4179 Nov 04 22:50:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4608(8156187) WEXITSTATUS 1 Nov 04 22:50:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4608(8156187) done Nov 04 22:51:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4656(8156318) WEXITSTATUS 1 Nov 04 22:51:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4656(8156318) done Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:51:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4557(8156014) WEXITSTATUS 1 Nov 04 22:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4557(8156014) done Nov 04 22:51:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158751 InitPrio=17453 usec=4631 Nov 04 22:51:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158751 NodeList=r806u23n03 #CPUs=2 Partition=devel Nov 04 22:51:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_452(7572620) Nov 04 22:51:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_458(7572626) Nov 04 22:51:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_470(7572638) Nov 04 22:51:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1854(8151180) WEXITSTATUS 0 Nov 04 22:51:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1854(8151180) done Nov 04 22:51:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2223(8152015) WEXITSTATUS 0 Nov 04 22:51:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2223(8152015) done Nov 04 22:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5253(8158677) WEXITSTATUS 1 Nov 04 22:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5253(8158677) done Nov 04 22:51:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158750 in day on r906u41n03 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_3(8158752) in day on r909u07n02 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_4(8158753) in day on r909u07n02 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_5(8158754) in day on r912u32n02 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3557(8158755) in gpu on r808u08n01 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3558(8158756) in gpu on r808u11n01 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3559(8158757) in gpu on r808u14n01 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3560(8158758) in gpu on r4513u20n01 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3561(8158759) in gpu on r4513u26n01 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 22:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157535 in scavenge on r815u29n05 Nov 04 22:51:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5099(8158432) WEXITSTATUS 1 Nov 04 22:51:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5099(8158432) done Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5298(8158760) in scavenge on r801u09n03 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5299(8158761) in scavenge on r801u27n02 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5300(8158762) in scavenge on r802u09n02 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5301(8158763) in scavenge on r802u11n03 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5302(8158764) in scavenge on r802u25n01 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5303(8158765) in scavenge on r808u08n01 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5304(8158766) in scavenge on r806u04n01 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5305(8158767) in scavenge on r912u32n02 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5306(8158768) in scavenge on r918u01n03 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5307(8158769) in scavenge on r909u07n02 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5308(8158770) in scavenge on r909u07n04 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5309(8158771) in scavenge on r909u13n04 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5310(8158772) in scavenge on r816u23n04 Nov 04 22:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5311(8158773) in scavenge on r816u29n01 Nov 04 22:52:07 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2266912548]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 04 22:52:07 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2266912548]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 04 22:52:11 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 22:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3559(8158757) WEXITSTATUS 0 Nov 04 22:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3559(8158757) done Nov 04 22:52:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4897(8158030) WEXITSTATUS 1 Nov 04 22:52:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4897(8158030) done Nov 04 22:52:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3560(8158758) WEXITSTATUS 0 Nov 04 22:52:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3560(8158758) done Nov 04 22:52:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158774 InitPrio=50679 usec=7857 Nov 04 22:52:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:52:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158774 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 22:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156042 WEXITSTATUS 0 Nov 04 22:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156042 done Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:53:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:53:15 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157536 NodeList=r816u15n11 #CPUs=12 Partition=scavenge Nov 04 22:53:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158749 uid 21916 Nov 04 22:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5280(8158722) WEXITSTATUS 1 Nov 04 22:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5280(8158722) done Nov 04 22:53:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4893(8158004) WEXITSTATUS 1 Nov 04 22:53:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4893(8158004) done Nov 04 22:53:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4830(8157809) WEXITSTATUS 1 Nov 04 22:53:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4830(8157809) done Nov 04 22:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5166(8158520) WEXITSTATUS 1 Nov 04 22:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5166(8158520) done Nov 04 22:53:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5168(8158522) WEXITSTATUS 1 Nov 04 22:53:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5168(8158522) done Nov 04 22:53:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_942(7573111) Nov 04 22:53:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_13(8151448) Nov 04 22:53:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158302 uid 26393 Nov 04 22:53:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2382(8152426) WEXITSTATUS 0 Nov 04 22:53:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2382(8152426) done Nov 04 22:53:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158775 InitPrio=72991 usec=6853 Nov 04 22:53:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158775 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 04 22:53:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5015(8158306) WEXITSTATUS 1 Nov 04 22:53:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5015(8158306) done Nov 04 22:53:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5277(8158719) WEXITSTATUS 1 Nov 04 22:53:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5277(8158719) done Nov 04 22:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4965(8158215) WEXITSTATUS 1 Nov 04 22:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4965(8158215) done Nov 04 22:54:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158776 InitPrio=10837 usec=2538 Nov 04 22:54:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158777 InitPrio=10837 usec=4201 Nov 04 22:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158656 WEXITSTATUS 0 Nov 04 22:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158656 done Nov 04 22:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2315(8152232) WEXITSTATUS 0 Nov 04 22:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2315(8152232) done Nov 04 22:54:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:54:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:54:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 22:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_28(8156151) WEXITSTATUS 0 Nov 04 22:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_28(8156151) done Nov 04 22:54:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158778 InitPrio=48651 usec=13670 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_1(8158779) in pi_ohern on r917u13n02 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_2(8158780) in pi_ohern on r917u13n02 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_3(8158781) in pi_ohern on r917u13n02 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_4(8158782) in pi_ohern on r917u13n02 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_5(8158783) in pi_ohern on r917u13n02 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_6(8158784) in pi_ohern on r917u13n02 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_7(8158785) in pi_ohern on r917u13n02 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_8(8158786) in pi_ohern on r917u29n03 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_9(8158787) in pi_ohern on r917u29n03 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_10(8158788) in pi_ohern on r917u29n03 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_11(8158789) in pi_ohern on r917u29n03 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_12(8158790) in pi_ohern on r917u29n03 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_13(8158791) in pi_ohern on r917u29n04 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_14(8158792) in pi_ohern on r917u29n04 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_15(8158793) in pi_ohern on r917u29n04 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_16(8158794) in pi_ohern on r917u29n04 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_17(8158795) in pi_ohern on r917u29n04 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_18(8158796) in pi_ohern on r917u29n04 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_19(8158797) in pi_ohern on r917u29n04 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_20(8158798) in pi_ohern on r917u29n04 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_21(8158799) in pi_ohern on r815u03n11 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_22(8158800) in pi_ohern on r815u03n11 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_23(8158801) in pi_ohern on r815u03n11 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_24(8158802) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_25(8158803) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_26(8158804) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_27(8158805) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_28(8158806) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_29(8158807) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_30(8158808) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_31(8158809) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_32(8158810) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_33(8158811) in pi_ohern on r815u03n12 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_34(8158812) in pi_ohern on r911u32n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_35(8158813) in pi_ohern on r911u32n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_36(8158814) in pi_ohern on r911u32n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_37(8158815) in pi_ohern on r911u32n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_38(8158816) in pi_ohern on r807u22n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_39(8158817) in pi_ohern on r807u24n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_40(8158818) in pi_ohern on r807u24n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_41(8158819) in pi_ohern on r807u24n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_42(8158820) in pi_ohern on r807u24n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_43(8158821) in pi_ohern on r903u12n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_44(8158822) in pi_ohern on r903u12n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_45(8158823) in pi_ohern on r903u12n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_46(8158824) in pi_ohern on r903u12n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_47(8158825) in pi_ohern on r903u26n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_48(8158826) in pi_ohern on r903u26n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_49(8158827) in pi_ohern on r903u26n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_50(8158828) in pi_ohern on r903u26n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_51(8158829) in pi_ohern on r913u32n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_52(8158830) in pi_ohern on r913u32n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_53(8158831) in pi_ohern on r913u32n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_54(8158832) in pi_ohern on r913u32n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_55(8158833) in pi_ohern on r913u35n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_56(8158834) in pi_ohern on r913u35n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_57(8158835) in pi_ohern on r913u35n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_58(8158836) in pi_ohern on r913u35n01 Nov 04 22:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_6(8158837) in day on r806u11n03 Nov 04 22:54:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3562(8158838) in gpu on r808u14n01 Nov 04 22:54:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3563(8158839) in gpu on r4513u20n01 Nov 04 22:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_605(7570267) WEXITSTATUS 0 Nov 04 22:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_605(7570267) done Nov 04 22:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_775(7570437) WEXITSTATUS 0 Nov 04 22:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_775(7570437) done Nov 04 22:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1043(7570705) WEXITSTATUS 0 Nov 04 22:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1043(7570705) done Nov 04 22:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_997(7570659) WEXITSTATUS 0 Nov 04 22:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_997(7570659) done Nov 04 22:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_155(7572323) WEXITSTATUS 0 Nov 04 22:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_155(7572323) done Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157800 WEXITSTATUS 0 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157800 done Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5312(8158840) in scavenge on r907u01n04 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5313(8158841) in scavenge on r802u11n02 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5314(8158842) in scavenge on r916u04n04 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5315(8158843) in scavenge on r917u07n04 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5316(8158844) in scavenge on r913u28n04 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5317(8158845) in scavenge on r805u02n04 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5318(8158846) in scavenge on r805u04n02 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5319(8158847) in scavenge on r909u07n03 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5320(8158848) in scavenge on r909u10n01 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5321(8158849) in scavenge on r909u13n04 Nov 04 22:55:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5322(8158850) in scavenge on r806u37n01 Nov 04 22:55:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4955(8158205) WEXITSTATUS 1 Nov 04 22:55:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4955(8158205) done Nov 04 22:55:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1848(8158851) in scavenge on r801u14n02 Nov 04 22:55:16 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4209(8152047) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:55:16 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4709(8156592) has been requeued to reclaim resources for JobId=Invalid Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_1(8158852) in scavenge on r908u28n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_2(8158853) in scavenge on r906u01n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_3(8158854) in scavenge on r906u01n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_4(8158855) in scavenge on r906u01n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_5(8158856) in scavenge on r908u39n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_6(8158857) in scavenge on r908u39n03 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_7(8158858) in scavenge on r908u33n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_8(8158859) in scavenge on r908u33n03 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_9(8158860) in scavenge on r908u33n04 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158776_10(8158776) in scavenge on r801u30n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2000(8151560) WEXITSTATUS 0 Nov 04 22:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2000(8151560) done Nov 04 22:55:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4709(8156592) Nov 04 22:55:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4209(8152047) Nov 04 22:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156045 WEXITSTATUS 0 Nov 04 22:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156045 done Nov 04 22:55:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158861 InitPrio=36433 usec=6666 Nov 04 22:55:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158861 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 22:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4891(8158002) WEXITSTATUS 1 Nov 04 22:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4891(8158002) done Nov 04 22:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5297(8158745) WEXITSTATUS 1 Nov 04 22:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5297(8158745) done Nov 04 22:55:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4216(8152271) WEXITSTATUS 1 Nov 04 22:55:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4216(8152271) done Nov 04 22:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158778 NodeList=r805u34n01 #CPUs=16 Partition=gpu Nov 04 22:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157537 NodeList=r815u23n07 #CPUs=12 Partition=scavenge Nov 04 22:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2183(8151975) WEXITSTATUS 0 Nov 04 22:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2183(8151975) done Nov 04 22:56:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156044 WEXITSTATUS 0 Nov 04 22:56:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156044 done Nov 04 22:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4933(8158120) WEXITSTATUS 1 Nov 04 22:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4933(8158120) done Nov 04 22:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5121(8158461) WEXITSTATUS 1 Nov 04 22:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5121(8158461) done Nov 04 22:56:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907376 WEXITSTATUS 0 Nov 04 22:56:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907376 done Nov 04 22:56:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5289(8158737) WEXITSTATUS 1 Nov 04 22:56:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5289(8158737) done Nov 04 22:56:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2288(8152205) WEXITSTATUS 0 Nov 04 22:56:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2288(8152205) done Nov 04 22:56:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1754(8150965) WEXITSTATUS 0 Nov 04 22:56:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1754(8150965) done Nov 04 22:57:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2115(8151791) WEXITSTATUS 0 Nov 04 22:57:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2115(8151791) done Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:57:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:57:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2245(8152037) WEXITSTATUS 0 Nov 04 22:57:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2245(8152037) done Nov 04 22:57:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3551(8158690) WEXITSTATUS 0 Nov 04 22:57:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3551(8158690) done Nov 04 22:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4988(8158261) WEXITSTATUS 1 Nov 04 22:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4988(8158261) done Nov 04 22:57:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4887(8157998) WEXITSTATUS 1 Nov 04 22:57:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4887(8157998) done Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_59(8158862) in pi_ohern on r917u29n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_60(8158863) in pi_ohern on r917u29n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_61(8158864) in pi_ohern on r917u29n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_62(8158865) in pi_ohern on r917u29n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_63(8158866) in pi_ohern on r917u29n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_64(8158867) in pi_ohern on r917u29n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_65(8158868) in pi_ohern on r917u29n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_66(8158869) in pi_ohern on r917u29n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_67(8158870) in pi_ohern on r807u24n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_68(8158871) in pi_ohern on r807u24n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_69(8158872) in pi_ohern on r807u24n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_70(8158873) in pi_ohern on r807u24n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_71(8158874) in pi_ohern on r807u24n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_72(8158875) in pi_ohern on r807u24n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_73(8158876) in pi_ohern on r807u24n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_74(8158877) in pi_ohern on r807u24n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_75(8158878) in pi_ohern on r903u26n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_76(8158879) in pi_ohern on r903u26n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_77(8158880) in pi_ohern on r903u26n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_78(8158881) in pi_ohern on r903u26n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_79(8158882) in pi_ohern on r903u26n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_80(8158883) in pi_ohern on r903u26n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_81(8158884) in pi_ohern on r903u26n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_82(8158885) in pi_ohern on r903u26n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_83(8158886) in pi_ohern on r913u35n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_84(8158887) in pi_ohern on r913u35n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_85(8158888) in pi_ohern on r913u35n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_86(8158889) in pi_ohern on r913u35n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_87(8158890) in pi_ohern on r913u35n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_88(8158891) in pi_ohern on r913u35n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_89(8158892) in pi_ohern on r913u35n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_90(8158893) in pi_ohern on r913u35n01 Nov 04 22:57:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_7(8158894) in day on r805u14n03 Nov 04 22:57:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907377 in gpu on r805u38n01 Nov 04 22:57:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3564(8158895) in gpu on r808u08n01 Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1033(7570695) WEXITSTATUS 0 Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1033(7570695) done Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_360(7571325) WEXITSTATUS 0 Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_360(7571325) done Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_923(7571889) WEXITSTATUS 0 Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_923(7571889) done Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1053(7570715) WEXITSTATUS 0 Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1053(7570715) done Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_212(7572380) WEXITSTATUS 0 Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_212(7572380) done Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_957(7573126) WEXITSTATUS 0 Nov 04 22:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_957(7573126) done Nov 04 22:57:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n01 Nov 04 22:57:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1516(8150361) WEXITSTATUS 0 Nov 04 22:57:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1516(8150361) done Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5323(8158896) in scavenge on r911u05n02 Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5324(8158897) in scavenge on r801u04n03 Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5325(8158898) in scavenge on r801u23n01 Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5326(8158899) in scavenge on r815u23n07 Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5327(8158900) in scavenge on r806u04n03 Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5328(8158901) in scavenge on r914u05n04 Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5329(8158902) in scavenge on r919u01n01 Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5330(8158903) in scavenge on r919u09n02 Nov 04 22:58:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5331(8158904) in scavenge on r919u36n01 Nov 04 22:58:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5293(8158741) WEXITSTATUS 1 Nov 04 22:58:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5293(8158741) done Nov 04 22:58:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3811(8144425) WEXITSTATUS 1 Nov 04 22:58:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3811(8144425) done Nov 04 22:58:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158905 InitPrio=10837 usec=2212 Nov 04 22:58:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158906 InitPrio=10837 usec=4684 Nov 04 22:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158654 WEXITSTATUS 0 Nov 04 22:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158654 done Nov 04 22:58:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5218(8158605) WEXITSTATUS 1 Nov 04 22:58:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5218(8158605) done Nov 04 22:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2098(8151774) WEXITSTATUS 0 Nov 04 22:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2098(8151774) done Nov 04 22:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4766(8157337) WEXITSTATUS 1 Nov 04 22:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4766(8157337) done Nov 04 22:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5251(8158675) WEXITSTATUS 1 Nov 04 22:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5251(8158675) done Nov 04 22:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3562(8158838) WEXITSTATUS 0 Nov 04 22:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3562(8158838) done Nov 04 22:59:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 22:59:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 22:59:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4910(8158073) WEXITSTATUS 1 Nov 04 22:59:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4910(8158073) done Nov 04 22:59:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8147452 Nov 04 23:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 04 23:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 04 23:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 04 23:00:02 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 04 23:00:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 04 23:00:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 04 23:00:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3525(8158254) WEXITSTATUS 0 Nov 04 23:00:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3525(8158254) done Nov 04 23:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156046 WEXITSTATUS 0 Nov 04 23:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156046 done Nov 04 23:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5049(8158345) WEXITSTATUS 1 Nov 04 23:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5049(8158345) done Nov 04 23:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4873(8157977) WEXITSTATUS 1 Nov 04 23:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4873(8157977) done Nov 04 23:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4411(8155449) WEXITSTATUS 1 Nov 04 23:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4411(8155449) done Nov 04 23:00:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4241(8152908) WEXITSTATUS 1 Nov 04 23:00:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4241(8152908) done Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_91(8158907) in pi_ohern on r917u13n02 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_92(8158908) in pi_ohern on r807u24n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_93(8158909) in pi_ohern on r807u24n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_94(8158910) in pi_ohern on r807u24n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_95(8158911) in pi_ohern on r807u24n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_96(8158912) in pi_ohern on r807u24n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_97(8158913) in pi_ohern on r807u24n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_98(8158914) in pi_ohern on r807u24n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_99(8158915) in pi_ohern on r807u24n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_100(8158916) in pi_ohern on r903u12n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_101(8158917) in pi_ohern on r903u12n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_102(8158918) in pi_ohern on r903u12n01 Nov 04 23:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_103(8158919) in pi_ohern on r903u12n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_104(8158920) in pi_ohern on r903u12n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_105(8158921) in pi_ohern on r903u12n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_106(8158922) in pi_ohern on r903u12n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_107(8158923) in pi_ohern on r903u12n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_108(8158924) in pi_ohern on r903u26n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_109(8158925) in pi_ohern on r903u26n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_110(8158926) in pi_ohern on r903u26n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_111(8158927) in pi_ohern on r903u26n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_112(8158928) in pi_ohern on r903u26n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_113(8158929) in pi_ohern on r903u26n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_114(8158930) in pi_ohern on r903u26n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_115(8158931) in pi_ohern on r903u26n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_116(8158932) in pi_ohern on r913u32n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_117(8158933) in pi_ohern on r913u32n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_118(8158934) in pi_ohern on r913u32n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_119(8158935) in pi_ohern on r913u32n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_120(8158936) in pi_ohern on r913u32n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_121(8158937) in pi_ohern on r913u32n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_122(8158938) in pi_ohern on r913u32n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_123(8158939) in pi_ohern on r913u32n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_124(8158940) in pi_ohern on r913u35n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_125(8158941) in pi_ohern on r913u35n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_126(8158942) in pi_ohern on r913u35n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_127(8158943) in pi_ohern on r913u35n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_128(8158944) in pi_ohern on r913u35n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_129(8158945) in pi_ohern on r913u35n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_130(8158946) in pi_ohern on r913u35n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_131(8158947) in pi_ohern on r913u35n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n03 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_8(8158948) in day on r912u36n04 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3565(8158949) in gpu on r808u14n01 Nov 04 23:00:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3566(8158950) in gpu on r903u34n01 Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_115(7571079) WEXITSTATUS 0 Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_115(7571079) done Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_275(7572443) WEXITSTATUS 0 Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_275(7572443) done Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_786(8115055) WEXITSTATUS 0 Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_786(8115055) done Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_53(7572220) WEXITSTATUS 0 Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_53(7572220) done Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_171(8114437) WEXITSTATUS 0 Nov 04 23:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_171(8114437) done Nov 04 23:00:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157538 in scavenge on r816u29n11 Nov 04 23:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5319(8158847) WEXITSTATUS 1 Nov 04 23:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5319(8158847) done Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5332(8158951) in scavenge on r907u05n01 Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5333(8158952) in scavenge on r802u30n03 Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5334(8158953) in scavenge on r917u10n03 Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5335(8158954) in scavenge on r815u09n11 Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5336(8158955) in scavenge on r909u01n01 Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5337(8158956) in scavenge on r919u09n02 Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5338(8158957) in scavenge on r919u32n04 Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5339(8158958) in scavenge on r815u15n02 Nov 04 23:01:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5340(8158959) in scavenge on r816u03n04 Nov 04 23:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 04 23:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 04 23:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1387(8123618) WEXITSTATUS 1 Nov 04 23:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1387(8123618) done Nov 04 23:01:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 04 23:01:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3565(8158949) WEXITSTATUS 0 Nov 04 23:01:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3565(8158949) done Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_1(8158960) in scavenge on r908u28n03 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_2(8158961) in scavenge on r908u37n02 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_3(8158962) in scavenge on r801u30n01 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_4(8158963) in scavenge on r801u30n01 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_5(8158964) in scavenge on r801u30n02 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_6(8158965) in scavenge on r801u30n02 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_7(8158966) in scavenge on r801u30n03 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_8(8158967) in scavenge on r801u30n03 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_9(8158968) in scavenge on r801u30n03 Nov 04 23:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158905_10(8158905) in scavenge on r801u30n04 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:01:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4989(8158262) WEXITSTATUS 1 Nov 04 23:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4989(8158262) done Nov 04 23:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1099(8131591) WEXITSTATUS 0 Nov 04 23:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1099(8131591) done Nov 04 23:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1530(8150509) WEXITSTATUS 0 Nov 04 23:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1530(8150509) done Nov 04 23:01:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2058(8125845) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:01:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2058(8125845) Nov 04 23:02:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8158969 InitPrio=36320 usec=3508 Nov 04 23:02:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1957(8151395) WEXITSTATUS 0 Nov 04 23:02:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1957(8151395) done Nov 04 23:02:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158969 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 23:02:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157539 NodeList=r816u29n10 #CPUs=12 Partition=scavenge Nov 04 23:02:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8158970 NodeList=(null) usec=21827 Nov 04 23:02:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158970 NodeList=r802u25n04 #CPUs=1 Partition=pi_econ_lp Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_25(8156148) WEXITSTATUS 0 Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8156148: assoc 5273 TRES cpu grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8156148: assoc 5273 TRES mem grp_used_tres_run_secs underflow, tried to remove 171769856 seconds when only 171524096 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8156148: assoc 5273 TRES cpu grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8156148: assoc 5273 TRES node grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8156148: assoc 5273 TRES billing grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8156148: assoc 246 TRES cpu grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8156148: assoc 246 TRES mem grp_used_tres_run_secs underflow, tried to remove 171769856 seconds when only 171524096 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8156148: assoc 5273 TRES mem grp_used_tres_run_secs underflow, tried to remove 171769856 seconds when only 171524096 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8156148: assoc 246 TRES node grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8156148: assoc 246 TRES billing grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8156148: assoc 5273 TRES node grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8156148: assoc 5273 TRES billing grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8156148: assoc 246 TRES cpu grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8156148: assoc 246 TRES mem grp_used_tres_run_secs underflow, tried to remove 171769856 seconds when only 171524096 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8156148: assoc 246 TRES node grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8156148: assoc 246 TRES billing grp_used_tres_run_secs underflow, tried to remove 10484 seconds when only 10424 remained. Nov 04 23:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156123_25(8156148) done Nov 04 23:02:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5299(8158761) WEXITSTATUS 1 Nov 04 23:02:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5299(8158761) done Nov 04 23:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5213(8158600) WEXITSTATUS 1 Nov 04 23:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5213(8158600) done Nov 04 23:03:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4797(8157693) WEXITSTATUS 1 Nov 04 23:03:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4797(8157693) done Nov 04 23:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2019(8151579) WEXITSTATUS 0 Nov 04 23:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2019(8151579) done Nov 04 23:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4511(8155900) WEXITSTATUS 1 Nov 04 23:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4511(8155900) done Nov 04 23:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2297(8152214) WEXITSTATUS 0 Nov 04 23:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2297(8152214) done Nov 04 23:03:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3558(8158756) WEXITSTATUS 0 Nov 04 23:03:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3558(8158756) done Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:03:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3566(8158950) WEXITSTATUS 0 Nov 04 23:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3566(8158950) done Nov 04 23:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5254(8158678) WEXITSTATUS 1 Nov 04 23:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5254(8158678) done Nov 04 23:03:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4595(8156116) WEXITSTATUS 1 Nov 04 23:03:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4595(8156116) done Nov 04 23:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4872(8157976) WEXITSTATUS 1 Nov 04 23:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4872(8157976) done Nov 04 23:03:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2398(8152442) WEXITSTATUS 0 Nov 04 23:03:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2398(8152442) done Nov 04 23:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4722(8156733) WEXITSTATUS 1 Nov 04 23:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4722(8156733) done Nov 04 23:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_132(8158971) in pi_ohern on r815u03n12 Nov 04 23:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_133(8158972) in pi_ohern on r815u03n12 Nov 04 23:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_134(8158973) in pi_ohern on r815u03n12 Nov 04 23:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_135(8158974) in pi_ohern on r815u03n12 Nov 04 23:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_136(8158975) in pi_ohern on r815u03n12 Nov 04 23:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_137(8158976) in pi_ohern on r815u03n12 Nov 04 23:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_138(8158977) in pi_ohern on r815u03n12 Nov 04 23:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_139(8158978) in pi_ohern on r815u03n12 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_140(8158979) in pi_ohern on r911u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_141(8158980) in pi_ohern on r911u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_142(8158981) in pi_ohern on r911u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_143(8158982) in pi_ohern on r911u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_144(8158983) in pi_ohern on r911u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_145(8158984) in pi_ohern on r911u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_146(8158985) in pi_ohern on r911u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_147(8158986) in pi_ohern on r911u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_148(8158987) in pi_ohern on r913u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_149(8158988) in pi_ohern on r913u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_150(8158989) in pi_ohern on r913u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_151(8158990) in pi_ohern on r913u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_152(8158991) in pi_ohern on r913u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_153(8158992) in pi_ohern on r913u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_154(8158993) in pi_ohern on r913u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_155(8158994) in pi_ohern on r913u32n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_156(8158995) in pi_ohern on r903u12n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_157(8158996) in pi_ohern on r903u12n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_158(8158997) in pi_ohern on r903u12n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_159(8158998) in pi_ohern on r903u12n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_160(8158999) in pi_ohern on r903u12n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_161(8159000) in pi_ohern on r903u12n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_162(8159001) in pi_ohern on r903u12n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_163(8159002) in pi_ohern on r903u12n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_164(8159003) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_165(8159004) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_166(8159005) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_167(8159006) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_168(8159007) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_169(8159008) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_170(8159009) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_171(8159010) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_172(8159011) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_173(8159012) in pi_ohern on r903u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_9(8159013) in day on r913u24n01 Nov 04 23:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_10(8159014) in day on r816u09n08 Nov 04 23:04:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3567(8159015) in gpu on r903u34n01 Nov 04 23:04:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3568(8159016) in gpu on r808u11n01 Nov 04 23:04:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3569(8159017) in gpu on r808u14n01 Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_72(7572239) WEXITSTATUS 0 Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_72(7572239) done Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_475(7572643) WEXITSTATUS 0 Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_475(7572643) done Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_89(7572256) WEXITSTATUS 0 Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_89(7572256) done Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_438(7572606) WEXITSTATUS 0 Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_438(7572606) done Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_434(7572602) WEXITSTATUS 0 Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_434(7572602) done Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1156(7572123) WEXITSTATUS 0 Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1156(7572123) done Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_658(7571623) WEXITSTATUS 0 Nov 04 23:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_658(7571623) done Nov 04 23:04:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n01 Nov 04 23:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2184(8151976) WEXITSTATUS 0 Nov 04 23:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2184(8151976) done Nov 04 23:04:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2392(8152436) WEXITSTATUS 0 Nov 04 23:04:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2392(8152436) done Nov 04 23:04:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159018 InitPrio=40248 usec=8779 Nov 04 23:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5341(8159019) in scavenge on r801u27n02 Nov 04 23:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5342(8159020) in scavenge on r802u11n01 Nov 04 23:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5343(8159021) in scavenge on r802u11n03 Nov 04 23:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5344(8159022) in scavenge on r906u41n03 Nov 04 23:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5345(8159023) in scavenge on r909u07n03 Nov 04 23:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5346(8159024) in scavenge on r914u05n01 Nov 04 23:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5347(8159025) in scavenge on r913u24n04 Nov 04 23:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5348(8159026) in scavenge on r913u28n04 Nov 04 23:04:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3569(8159017) WEXITSTATUS 0 Nov 04 23:04:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3569(8159017) done Nov 04 23:04:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159018 NodeList=r904u07n02 #CPUs=8 Partition=day Nov 04 23:04:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1849(8159027) in scavenge on r801u09n04 Nov 04 23:04:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2396(8152440) WEXITSTATUS 0 Nov 04 23:04:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2396(8152440) done Nov 04 23:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5146(8158493) WEXITSTATUS 1 Nov 04 23:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5146(8158493) done Nov 04 23:04:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159028 InitPrio=10837 usec=2339 Nov 04 23:04:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159029 InitPrio=10837 usec=3634 Nov 04 23:04:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158655 WEXITSTATUS 0 Nov 04 23:04:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158655 done Nov 04 23:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4823(8157802) WEXITSTATUS 1 Nov 04 23:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4823(8157802) done Nov 04 23:05:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3567(8159015) WEXITSTATUS 0 Nov 04 23:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3567(8159015) done Nov 04 23:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159018 WEXITSTATUS 1 Nov 04 23:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159018 done Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:05:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:05:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5291(8158739) WEXITSTATUS 1 Nov 04 23:05:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5291(8158739) done Nov 04 23:05:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4590(8156111) WEXITSTATUS 1 Nov 04 23:05:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4590(8156111) done Nov 04 23:05:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1905(8151231) WEXITSTATUS 0 Nov 04 23:05:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1905(8151231) done Nov 04 23:06:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4862(8157961) WEXITSTATUS 1 Nov 04 23:06:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4862(8157961) done Nov 04 23:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2189(8151981) WEXITSTATUS 0 Nov 04 23:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2189(8151981) done Nov 04 23:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3561(8158759) WEXITSTATUS 0 Nov 04 23:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3561(8158759) done Nov 04 23:06:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_273(7572441) Nov 04 23:06:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8157821 Nov 04 23:06:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4959(8158209) WEXITSTATUS 1 Nov 04 23:06:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4959(8158209) done Nov 04 23:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4802(8157748) WEXITSTATUS 1 Nov 04 23:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4802(8157748) done Nov 04 23:06:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4486(8155859) WEXITSTATUS 1 Nov 04 23:06:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4486(8155859) done Nov 04 23:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5247(8158671) WEXITSTATUS 1 Nov 04 23:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5247(8158671) done Nov 04 23:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5074(8158386) WEXITSTATUS 1 Nov 04 23:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5074(8158386) done Nov 04 23:06:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1293(8135299) WEXITSTATUS 0 Nov 04 23:06:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1293(8135299) done Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_174(8159030) in pi_ohern on r801u32n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_175(8159031) in pi_ohern on r801u32n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_176(8159032) in pi_ohern on r801u32n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_177(8159033) in pi_ohern on r801u32n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_178(8159034) in pi_ohern on r801u32n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_179(8159035) in pi_ohern on r801u32n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_180(8159036) in pi_ohern on r801u32n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_181(8159037) in pi_ohern on r801u32n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_182(8159038) in pi_ohern on r917u13n02 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_183(8159039) in pi_ohern on r917u13n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_184(8159040) in pi_ohern on r917u13n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_185(8159041) in pi_ohern on r917u13n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_186(8159042) in pi_ohern on r917u13n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_187(8159043) in pi_ohern on r917u13n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_188(8159044) in pi_ohern on r917u13n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_189(8159045) in pi_ohern on r917u13n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_190(8159046) in pi_ohern on r917u13n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_191(8159047) in pi_ohern on r917u29n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_192(8159048) in pi_ohern on r917u29n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_193(8159049) in pi_ohern on r917u29n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_194(8159050) in pi_ohern on r917u29n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_195(8159051) in pi_ohern on r917u29n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_196(8159052) in pi_ohern on r917u29n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_197(8159053) in pi_ohern on r917u29n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_198(8159054) in pi_ohern on r917u29n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_199(8159055) in pi_ohern on r917u29n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_200(8159056) in pi_ohern on r917u29n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_201(8159057) in pi_ohern on r917u29n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_202(8159058) in pi_ohern on r917u29n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_203(8159059) in pi_ohern on r917u29n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_204(8159060) in pi_ohern on r917u29n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_205(8159061) in pi_ohern on r917u29n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_206(8159062) in pi_ohern on r917u29n04 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_207(8159063) in pi_ohern on r815u03n11 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_208(8159064) in pi_ohern on r815u03n11 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_209(8159065) in pi_ohern on r815u03n11 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_210(8159066) in pi_ohern on r815u03n11 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_211(8159067) in pi_ohern on r815u03n11 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_212(8159068) in pi_ohern on r815u03n11 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_213(8159069) in pi_ohern on r911u32n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_214(8159070) in pi_ohern on r911u32n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_215(8159071) in pi_ohern on r911u32n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_216(8159072) in pi_ohern on r911u32n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_217(8159073) in pi_ohern on r911u32n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_218(8159074) in pi_ohern on r911u32n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_219(8159075) in pi_ohern on r911u32n01 Nov 04 23:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_220(8159076) in pi_ohern on r911u32n01 Nov 04 23:07:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_11(8159077) in day on r806u09n02 Nov 04 23:07:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_12(8159078) in day on r909u01n01 Nov 04 23:07:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3570(8159079) in gpu on r903u34n01 Nov 04 23:07:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3571(8159080) in gpu on r808u14n01 Nov 04 23:07:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3572(8159081) in gpu on r4513u26n01 Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_472(7572640) WEXITSTATUS 0 Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_472(7572640) done Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_157(7572325) WEXITSTATUS 0 Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_157(7572325) done Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_554(7572723) WEXITSTATUS 0 Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_554(7572723) done Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1169(7572136) WEXITSTATUS 0 Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1169(7572136) done Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_675(7572844) WEXITSTATUS 0 Nov 04 23:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_675(7572844) done Nov 04 23:07:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1850(8159082) in day on r815u09n06 Nov 04 23:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2034(8151594) WEXITSTATUS 0 Nov 04 23:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2034(8151594) done Nov 04 23:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3557(8158755) WEXITSTATUS 0 Nov 04 23:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3557(8158755) done Nov 04 23:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5133(8158473) WEXITSTATUS 1 Nov 04 23:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5133(8158473) done Nov 04 23:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5349(8159083) in scavenge on r907u05n02 Nov 04 23:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5350(8159084) in scavenge on r802u06n04 Nov 04 23:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5351(8159085) in scavenge on r916u04n01 Nov 04 23:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5352(8159086) in scavenge on r801u06n01 Nov 04 23:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5353(8159087) in scavenge on r801u06n02 Nov 04 23:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5354(8159088) in scavenge on r801u09n03 Nov 04 23:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5355(8159089) in scavenge on r912u28n01 Nov 04 23:07:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5356(8159090) in scavenge on r919u09n01 Nov 04 23:07:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5357(8159091) in scavenge on r807u08n02 Nov 04 23:07:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5358(8159092) in scavenge on r807u08n03 Nov 04 23:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5321(8158849) WEXITSTATUS 1 Nov 04 23:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5321(8158849) done Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_1(8159093) in scavenge on r801u30n04 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_2(8159094) in scavenge on r801u30n04 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_3(8159095) in scavenge on r907u01n02 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_4(8159096) in scavenge on r907u01n02 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_5(8159097) in scavenge on r907u01n02 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_6(8159098) in scavenge on r907u01n03 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_7(8159099) in scavenge on r907u01n03 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_8(8159100) in scavenge on r907u01n03 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_9(8159101) in scavenge on r907u01n04 Nov 04 23:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_10(8159028) in scavenge on r907u01n04 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3570(8159079) WEXITSTATUS 0 Nov 04 23:07:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3570(8159079) done Nov 04 23:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_10(8135709) WEXITSTATUS 0 Nov 04 23:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_10(8135709) done Nov 04 23:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5327(8158900) WEXITSTATUS 1 Nov 04 23:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5327(8158900) done Nov 04 23:07:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5323(8158896) WEXITSTATUS 1 Nov 04 23:07:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5323(8158896) done Nov 04 23:07:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5096(8158420) WEXITSTATUS 1 Nov 04 23:07:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5096(8158420) done Nov 04 23:07:53 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4812(8157769) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:07:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4812(8157769) Nov 04 23:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4808(8157754) WEXITSTATUS 1 Nov 04 23:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4808(8157754) done Nov 04 23:08:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5222(8158609) WEXITSTATUS 1 Nov 04 23:08:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5222(8158609) done Nov 04 23:08:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4695(8156467) WEXITSTATUS 1 Nov 04 23:08:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4695(8156467) done Nov 04 23:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4783(8157658) WEXITSTATUS 1 Nov 04 23:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4783(8157658) done Nov 04 23:08:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159102 InitPrio=22669 usec=2157 Nov 04 23:08:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159103 InitPrio=22669 usec=2577 Nov 04 23:08:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159104 InitPrio=22669 usec=1679 Nov 04 23:08:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159105 InitPrio=22669 usec=1712 Nov 04 23:08:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159106 InitPrio=22669 usec=1587 Nov 04 23:08:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159107 InitPrio=22669 usec=2525 Nov 04 23:08:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159108 InitPrio=22669 usec=1747 Nov 04 23:08:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159109 InitPrio=22669 usec=1560 Nov 04 23:08:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159110 InitPrio=22669 usec=2238 Nov 04 23:08:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159111 InitPrio=22669 usec=1591 Nov 04 23:08:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5044(8158340) WEXITSTATUS 1 Nov 04 23:08:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5044(8158340) done Nov 04 23:08:49 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8159112 NodeList=(null) usec=19576 Nov 04 23:08:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159112 NodeList=r802u25n04 #CPUs=1 Partition=pi_econ_lp Nov 04 23:08:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8143894_34(8144922) WEXITSTATUS 0 Nov 04 23:08:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8143894_34(8144922) done Nov 04 23:08:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5353(8159087) WEXITSTATUS 1 Nov 04 23:08:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5353(8159087) done Nov 04 23:08:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159113 InitPrio=22669 usec=2565 Nov 04 23:08:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159114 InitPrio=22669 usec=1852 Nov 04 23:08:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159115 InitPrio=22669 usec=2745 Nov 04 23:08:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159116 InitPrio=22669 usec=1922 Nov 04 23:08:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159117 InitPrio=22669 usec=1665 Nov 04 23:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1857(8151183) WEXITSTATUS 0 Nov 04 23:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1857(8151183) done Nov 04 23:09:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2386(8152430) WEXITSTATUS 0 Nov 04 23:09:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2386(8152430) done Nov 04 23:09:12 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25765: QOSMaxSubmitJobPerUserLimit Nov 04 23:09:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 23:09:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5326(8158899) WEXITSTATUS 1 Nov 04 23:09:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5326(8158899) done Nov 04 23:09:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5154(8158501) WEXITSTATUS 1 Nov 04 23:09:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5154(8158501) done Nov 04 23:09:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3546(8158641) WEXITSTATUS 0 Nov 04 23:09:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3546(8158641) done Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:09:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:09:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3572(8159081) WEXITSTATUS 0 Nov 04 23:09:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3572(8159081) done Nov 04 23:09:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159118 InitPrio=10837 usec=2237 Nov 04 23:09:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159119 InitPrio=10837 usec=4673 Nov 04 23:09:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158657 WEXITSTATUS 0 Nov 04 23:09:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158657 done Nov 04 23:09:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_4(8157552) WEXITSTATUS 0 Nov 04 23:09:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_4(8157552) done Nov 04 23:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4981(8158243) WEXITSTATUS 1 Nov 04 23:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4981(8158243) done Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_221(8159120) in pi_ohern on r801u32n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_222(8159121) in pi_ohern on r801u32n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_223(8159122) in pi_ohern on r801u32n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_224(8159123) in pi_ohern on r801u32n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_225(8159124) in pi_ohern on r801u32n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_226(8159125) in pi_ohern on r801u32n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_227(8159126) in pi_ohern on r801u32n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_228(8159127) in pi_ohern on r801u32n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_229(8159128) in pi_ohern on r917u13n02 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_230(8159129) in pi_ohern on r917u29n04 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_231(8159130) in pi_ohern on r917u29n04 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_232(8159131) in pi_ohern on r917u29n04 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_233(8159132) in pi_ohern on r917u29n04 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_234(8159133) in pi_ohern on r917u29n04 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_235(8159134) in pi_ohern on r917u29n04 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_236(8159135) in pi_ohern on r917u29n04 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_237(8159136) in pi_ohern on r917u29n04 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_238(8159137) in pi_ohern on r815u03n11 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_239(8159138) in pi_ohern on r815u03n11 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_240(8159139) in pi_ohern on r815u03n11 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_241(8159140) in pi_ohern on r815u03n11 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_242(8159141) in pi_ohern on r815u03n11 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_243(8159142) in pi_ohern on r815u03n11 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_244(8159143) in pi_ohern on r807u22n01 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_245(8159144) in pi_ohern on r807u22n01 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_246(8159145) in pi_ohern on r807u22n01 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_247(8159146) in pi_ohern on r807u22n01 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_248(8159147) in pi_ohern on r807u22n01 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_249(8159148) in pi_ohern on r807u22n01 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_250(8159149) in pi_ohern on r903u24n01 Nov 04 23:10:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_251(8159150) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_252(8159151) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_253(8159152) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_254(8159153) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_255(8159154) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_256(8159155) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_257(8159156) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_258(8159157) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_259(8159158) in pi_ohern on r903u24n01 Nov 04 23:10:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_13(8159159) in day on r806u09n03 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3573(8159160) in gpu on r808u08n01 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3574(8159161) in gpu on r4513u26n01 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3575(8159162) in gpu on r903u34n01 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3576(8159163) in gpu on r903u40n01 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_336(7572504) WEXITSTATUS 0 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_336(7572504) done Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_467(8114734) WEXITSTATUS 0 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_467(8114734) done Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_515(7572683) WEXITSTATUS 0 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_515(7572683) done Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_497(7572665) WEXITSTATUS 0 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_497(7572665) done Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_1143(8115412) WEXITSTATUS 0 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_1143(8115412) done Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_431(7572599) WEXITSTATUS 0 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_431(7572599) done Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_689(7572858) WEXITSTATUS 0 Nov 04 23:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_689(7572858) done Nov 04 23:10:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n01 Nov 04 23:10:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159164 InitPrio=22669 usec=2182 Nov 04 23:10:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159165 InitPrio=22669 usec=1843 Nov 04 23:10:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159166 InitPrio=22669 usec=2679 Nov 04 23:10:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159167 InitPrio=22669 usec=1713 Nov 04 23:10:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159168 InitPrio=22669 usec=1578 Nov 04 23:10:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156047 WEXITSTATUS 0 Nov 04 23:10:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156047 done Nov 04 23:10:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3576(8159163) WEXITSTATUS 0 Nov 04 23:10:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3576(8159163) done Nov 04 23:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157540 NodeList=r816u15n11 #CPUs=12 Partition=scavenge Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159102 in scavenge on r906u01n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159103 in scavenge on r801u30n01 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159104 in scavenge on r801u30n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159105 in scavenge on r801u30n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159106 in scavenge on r801u30n04 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159107 in scavenge on r908u33n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159108 in scavenge on r908u33n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159109 in scavenge on r908u33n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159110 in scavenge on r908u33n03 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159111 in scavenge on r908u33n03 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159113 in scavenge on r908u33n03 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159114 in scavenge on r908u33n04 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159115 in scavenge on r908u33n04 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159116 in scavenge on r908u33n04 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159117 in scavenge on r908u37n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5359(8159169) in scavenge on r801u06n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5360(8159170) in scavenge on r911u05n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5361(8159171) in scavenge on r916u01n01 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5362(8159172) in scavenge on r815u23n07 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5363(8159173) in scavenge on r907u32n01 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5364(8159174) in scavenge on r806u04n03 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5365(8159175) in scavenge on r806u23n02 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5366(8159176) in scavenge on r909u13n03 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5367(8159177) in scavenge on r909u13n04 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5368(8159178) in scavenge on r918u32n03 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5369(8159179) in scavenge on r919u01n03 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5370(8159180) in scavenge on r919u32n01 Nov 04 23:10:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5371(8159181) in scavenge on r815u15n07 Nov 04 23:10:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5097(8158421) WEXITSTATUS 1 Nov 04 23:10:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5097(8158421) done Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_1(8159182) in scavenge on r907u01n04 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_2(8159183) in scavenge on r907u05n01 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_3(8159184) in scavenge on r907u05n01 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_4(8159185) in scavenge on r907u05n01 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_5(8159186) in scavenge on r907u05n02 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_6(8159187) in scavenge on r907u05n02 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_7(8159188) in scavenge on r907u05n02 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_8(8159189) in scavenge on r907u05n03 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_9(8159190) in scavenge on r907u05n03 Nov 04 23:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_10(8159118) in scavenge on r907u05n03 Nov 04 23:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4749(8157099) WEXITSTATUS 1 Nov 04 23:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4749(8157099) done Nov 04 23:10:54 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1431(8135964) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:10:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159191 InitPrio=22669 usec=2391 Nov 04 23:10:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159192 InitPrio=22669 usec=1862 Nov 04 23:10:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159193 InitPrio=22669 usec=2303 Nov 04 23:10:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159194 InitPrio=22669 usec=1744 Nov 04 23:10:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159195 InitPrio=22669 usec=1403 Nov 04 23:10:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1431(8135964) Nov 04 23:11:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4191(8151646) WEXITSTATUS 1 Nov 04 23:11:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4191(8151646) done Nov 04 23:11:25 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:11:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159196 InitPrio=22669 usec=2418 Nov 04 23:11:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159197 InitPrio=22669 usec=1777 Nov 04 23:11:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159198 InitPrio=22669 usec=1696 Nov 04 23:11:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159199 InitPrio=22669 usec=2832 Nov 04 23:11:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159200 InitPrio=22669 usec=1764 Nov 04 23:11:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159201 InitPrio=22669 usec=2439 Nov 04 23:11:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159202 InitPrio=22669 usec=1583 Nov 04 23:11:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159203 InitPrio=22669 usec=1927 Nov 04 23:11:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159204 InitPrio=22669 usec=2777 Nov 04 23:11:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159205 InitPrio=22669 usec=1830 Nov 04 23:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5365(8159175) WEXITSTATUS 1 Nov 04 23:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5365(8159175) done Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:11:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159206 InitPrio=22669 usec=3042 Nov 04 23:11:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159207 InitPrio=22669 usec=1986 Nov 04 23:11:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159208 InitPrio=22669 usec=2532 Nov 04 23:11:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159209 InitPrio=22669 usec=1687 Nov 04 23:11:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159210 InitPrio=22669 usec=1507 Nov 04 23:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3486(8157869) WEXITSTATUS 0 Nov 04 23:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3486(8157869) done Nov 04 23:11:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5340(8158959) WEXITSTATUS 1 Nov 04 23:11:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5340(8158959) done Nov 04 23:11:54 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1927(8125268) Nov 04 23:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5304(8158766) WEXITSTATUS 1 Nov 04 23:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5304(8158766) done Nov 04 23:11:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159211 InitPrio=22669 usec=2218 Nov 04 23:11:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159212 InitPrio=22669 usec=1415 Nov 04 23:11:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159213 InitPrio=22669 usec=2532 Nov 04 23:11:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159214 InitPrio=22669 usec=1538 Nov 04 23:11:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159215 InitPrio=22669 usec=1386 Nov 04 23:12:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159216 InitPrio=22669 usec=2105 Nov 04 23:12:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159217 InitPrio=22669 usec=2502 Nov 04 23:12:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159218 InitPrio=22669 usec=1859 Nov 04 23:12:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159219 InitPrio=22669 usec=1522 Nov 04 23:12:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159220 InitPrio=22669 usec=1520 Nov 04 23:12:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4946(8158191) WEXITSTATUS 1 Nov 04 23:12:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4946(8158191) done Nov 04 23:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3573(8159160) WEXITSTATUS 0 Nov 04 23:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3573(8159160) done Nov 04 23:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4567(8156024) WEXITSTATUS 1 Nov 04 23:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4567(8156024) done Nov 04 23:12:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159221 InitPrio=22669 usec=2416 Nov 04 23:12:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159222 InitPrio=22669 usec=2621 Nov 04 23:12:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159223 InitPrio=22669 usec=1723 Nov 04 23:12:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159224 InitPrio=22669 usec=1666 Nov 04 23:12:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159225 InitPrio=22669 usec=1876 Nov 04 23:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 04 23:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 04 23:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5332(8158951) WEXITSTATUS 1 Nov 04 23:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5332(8158951) done Nov 04 23:12:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 04 23:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3563(8158839) WEXITSTATUS 0 Nov 04 23:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3563(8158839) done Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_260(8159226) in pi_ohern on r917u29n03 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_261(8159227) in pi_ohern on r917u29n03 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_262(8159228) in pi_ohern on r917u29n03 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_263(8159229) in pi_ohern on r917u29n03 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_264(8159230) in pi_ohern on r917u29n03 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_265(8159231) in pi_ohern on r917u29n03 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_266(8159232) in pi_ohern on r917u29n03 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_267(8159233) in pi_ohern on r917u29n03 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_268(8159234) in pi_ohern on r917u29n04 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_269(8159235) in pi_ohern on r917u29n04 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_270(8159236) in pi_ohern on r917u29n04 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_271(8159237) in pi_ohern on r917u29n04 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_272(8159238) in pi_ohern on r917u29n04 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_273(8159239) in pi_ohern on r917u29n04 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_274(8159240) in pi_ohern on r917u29n04 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_275(8159241) in pi_ohern on r917u29n04 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_276(8159242) in pi_ohern on r815u03n11 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_277(8159243) in pi_ohern on r815u03n11 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_278(8159244) in pi_ohern on r815u03n11 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_279(8159245) in pi_ohern on r815u03n11 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_280(8159246) in pi_ohern on r815u03n11 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_281(8159247) in pi_ohern on r815u03n11 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_282(8159248) in pi_ohern on r815u03n12 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_283(8159249) in pi_ohern on r815u03n12 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_284(8159250) in pi_ohern on r815u03n12 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_285(8159251) in pi_ohern on r815u03n12 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_286(8159252) in pi_ohern on r815u03n12 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_287(8159253) in pi_ohern on r815u03n12 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_288(8159254) in pi_ohern on r815u03n12 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_289(8159255) in pi_ohern on r815u03n12 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_290(8159256) in pi_ohern on r807u22n01 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_291(8159257) in pi_ohern on r807u22n01 Nov 04 23:13:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_292(8159258) in pi_ohern on r807u22n01 Nov 04 23:13:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_14(8159259) in day on r805u23n04 Nov 04 23:13:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_15(8159260) in day on r918u01n02 Nov 04 23:13:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3577(8159261) in gpu on r903u40n01 Nov 04 23:13:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3578(8159262) in gpu on r808u08n01 Nov 04 23:13:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3579(8159263) in gpu on r4513u20n01 Nov 04 23:13:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3580(8159264) in gpu on r4513u29n01 Nov 04 23:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_751(7572920) WEXITSTATUS 0 Nov 04 23:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_751(7572920) done Nov 04 23:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_1044(8115313) WEXITSTATUS 0 Nov 04 23:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_1044(8115313) done Nov 04 23:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_767(7572936) WEXITSTATUS 0 Nov 04 23:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_767(7572936) done Nov 04 23:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_415(7572583) WEXITSTATUS 0 Nov 04 23:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_415(7572583) done Nov 04 23:13:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4633(8156225) WEXITSTATUS 1 Nov 04 23:13:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4633(8156225) done Nov 04 23:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2088(8151764) WEXITSTATUS 0 Nov 04 23:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2088(8151764) done Nov 04 23:13:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2362(8152406) WEXITSTATUS 0 Nov 04 23:13:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2362(8152406) done Nov 04 23:13:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3571(8159080) WEXITSTATUS 0 Nov 04 23:13:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3571(8159080) done Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159164 in scavenge on r908u37n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159165 in scavenge on r908u39n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159166 in scavenge on r908u39n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159167 in scavenge on r908u39n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159168 in scavenge on r908u39n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159191 in scavenge on r908u39n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159192 in scavenge on r908u39n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159193 in scavenge on r908u39n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159194 in scavenge on r908u39n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159195 in scavenge on r908u39n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159196 in scavenge on r907u01n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159197 in scavenge on r907u01n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159198 in scavenge on r907u01n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159199 in scavenge on r907u05n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159200 in scavenge on r907u05n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159201 in scavenge on r907u05n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159202 in scavenge on r907u05n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159203 in scavenge on r907u05n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159204 in scavenge on r907u05n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159205 in scavenge on r907u05n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159206 in scavenge on r907u05n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159207 in scavenge on r907u05n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159208 in scavenge on r907u05n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159209 in scavenge on r907u05n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159210 in scavenge on r907u05n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159211 in scavenge on r907u09n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159212 in scavenge on r907u09n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159213 in scavenge on r907u09n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159214 in scavenge on r907u09n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159215 in scavenge on r907u09n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159216 in scavenge on r907u09n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159217 in scavenge on r907u09n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159218 in scavenge on r907u09n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159219 in scavenge on r908u35n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159220 in scavenge on r908u35n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159221 in scavenge on r908u35n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159222 in scavenge on r908u35n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159223 in scavenge on r908u35n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159224 in scavenge on r908u35n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159225 in scavenge on r908u35n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5372(8159265) in scavenge on r802u02n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5373(8159266) in scavenge on r802u11n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5374(8159267) in scavenge on r805u23n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5375(8159268) in scavenge on r906u41n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5376(8159269) in scavenge on r904u10n03 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5377(8159270) in scavenge on r806u04n01 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5378(8159271) in scavenge on r806u23n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5379(8159272) in scavenge on r816u03n04 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5380(8159273) in scavenge on r815u23n02 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5381(8159274) in scavenge on r815u35n05 Nov 04 23:13:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5382(8159275) in scavenge on r815u35n06 Nov 04 23:13:33 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:102 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 23:13:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156048 WEXITSTATUS 0 Nov 04 23:13:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156048 done Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:13:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:13:53 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_893(8128055) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:13:53 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5004(8158288) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:13:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5004(8158288) Nov 04 23:13:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_893(8128055) Nov 04 23:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3580(8159264) WEXITSTATUS 0 Nov 04 23:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3580(8159264) done Nov 04 23:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1264(8135182) WEXITSTATUS 0 Nov 04 23:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1264(8135182) done Nov 04 23:14:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_473(7572641) Nov 04 23:14:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_479(7572647) Nov 04 23:14:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_491(7572659) Nov 04 23:14:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_522(7572691) Nov 04 23:14:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_980(7573149) Nov 04 23:14:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157541 NodeList=r816u23n12 #CPUs=12 Partition=scavenge Nov 04 23:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4950(8158195) WEXITSTATUS 1 Nov 04 23:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4950(8158195) done Nov 04 23:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_12(8135711) WEXITSTATUS 0 Nov 04 23:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_12(8135711) done Nov 04 23:14:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156049 WEXITSTATUS 0 Nov 04 23:14:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156049 done Nov 04 23:14:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1432(8135965) WEXITSTATUS 0 Nov 04 23:14:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1432(8135965) done Nov 04 23:14:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5038(8158334) WEXITSTATUS 1 Nov 04 23:14:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5038(8158334) done Nov 04 23:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4466(8155783) WEXITSTATUS 1 Nov 04 23:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4466(8155783) done Nov 04 23:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4800(8157696) WEXITSTATUS 1 Nov 04 23:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4800(8157696) done Nov 04 23:15:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156052 WEXITSTATUS 0 Nov 04 23:15:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156052 done Nov 04 23:15:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5182(8158536) WEXITSTATUS 1 Nov 04 23:15:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5182(8158536) done Nov 04 23:15:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5030(8158326) WEXITSTATUS 1 Nov 04 23:15:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5030(8158326) done Nov 04 23:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5246(8158653) WEXITSTATUS 1 Nov 04 23:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5246(8158653) done Nov 04 23:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2369(8152413) WEXITSTATUS 0 Nov 04 23:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2369(8152413) done Nov 04 23:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2758(8128232) WEXITSTATUS 1 Nov 04 23:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2758(8128232) done Nov 04 23:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1101(8131593) WEXITSTATUS 0 Nov 04 23:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1101(8131593) done Nov 04 23:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 24543: QOSMaxCpuPerUserLimit Nov 04 23:15:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 23:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3554(8158717) WEXITSTATUS 0 Nov 04 23:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3554(8158717) done Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3577(8159261) WEXITSTATUS 0 Nov 04 23:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3577(8159261) done Nov 04 23:15:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158454 Nov 04 23:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 24543: QOSMaxCpuPerUserLimit Nov 04 23:15:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 23:16:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8159276 NodeList=(null) usec=22152 Nov 04 23:16:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159276 NodeList=r806u23n04 #CPUs=4 Partition=devel Nov 04 23:16:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 23:16:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4170(8151030) WEXITSTATUS 1 Nov 04 23:16:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4170(8151030) done Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_293(8159277) in pi_ohern on r917u29n01 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_294(8159278) in pi_ohern on r917u29n01 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_295(8159279) in pi_ohern on r917u29n01 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_296(8159280) in pi_ohern on r917u29n01 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_297(8159281) in pi_ohern on r917u29n01 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_298(8159282) in pi_ohern on r917u29n01 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_299(8159283) in pi_ohern on r917u29n01 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_300(8159284) in pi_ohern on r917u29n01 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_301(8159285) in pi_ohern on r917u29n03 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_302(8159286) in pi_ohern on r917u29n03 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_303(8159287) in pi_ohern on r917u29n03 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_304(8159288) in pi_ohern on r917u29n03 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_305(8159289) in pi_ohern on r917u29n03 Nov 04 23:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_306(8159290) in pi_ohern on r917u29n03 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_307(8159291) in pi_ohern on r917u29n03 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_308(8159292) in pi_ohern on r917u29n03 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_309(8159293) in pi_ohern on r917u29n04 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_310(8159294) in pi_ohern on r917u29n04 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_311(8159295) in pi_ohern on r917u29n04 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_312(8159296) in pi_ohern on r917u29n04 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_313(8159297) in pi_ohern on r917u29n04 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_314(8159298) in pi_ohern on r917u29n04 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_315(8159299) in pi_ohern on r917u29n04 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_316(8159300) in pi_ohern on r917u29n04 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_317(8159301) in pi_ohern on r815u03n11 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_318(8159302) in pi_ohern on r815u03n11 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_319(8159303) in pi_ohern on r815u03n11 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_320(8159304) in pi_ohern on r815u03n11 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_321(8159305) in pi_ohern on r815u03n11 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_322(8159306) in pi_ohern on r815u03n12 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_323(8159307) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_324(8159308) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_325(8159309) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_326(8159310) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_327(8159311) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_328(8159312) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_329(8159313) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_330(8159314) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_331(8159315) in pi_ohern on r807u22n01 Nov 04 23:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_332(8159316) in pi_ohern on r807u22n01 Nov 04 23:16:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3581(8159317) in gpu on r903u40n01 Nov 04 23:16:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3582(8159318) in gpu on r808u14n01 Nov 04 23:16:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3583(8159319) in gpu on r808u17n01 Nov 04 23:16:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3584(8159320) in gpu on r4513u29n01 Nov 04 23:16:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 23:16:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157542 in scavenge on r815u29n04 Nov 04 23:16:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157543 in scavenge on r816u03n04 Nov 04 23:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_545(7572714) WEXITSTATUS 0 Nov 04 23:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_545(7572714) done Nov 04 23:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_818(7572987) WEXITSTATUS 0 Nov 04 23:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_818(7572987) done Nov 04 23:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_447(8114714) WEXITSTATUS 0 Nov 04 23:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_447(8114714) done Nov 04 23:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_905(7573074) WEXITSTATUS 0 Nov 04 23:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_905(7573074) done Nov 04 23:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5025(8158316) WEXITSTATUS 1 Nov 04 23:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5025(8158316) done Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5383(8159321) in scavenge on r801u02n04 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5384(8159322) in scavenge on r911u24n03 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5385(8159323) in scavenge on r802u02n02 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5386(8159324) in scavenge on r802u30n01 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5387(8159325) in scavenge on r805u04n01 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5388(8159326) in scavenge on r805u04n01 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5389(8159327) in scavenge on r909u29n04 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5390(8159328) in scavenge on r912u28n04 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5391(8159329) in scavenge on r912u36n03 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5392(8159330) in scavenge on r919u01n04 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5393(8159331) in scavenge on r919u01n04 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5394(8159332) in scavenge on r919u01n04 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5395(8159333) in scavenge on r919u05n04 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5396(8159334) in scavenge on r806u02n02 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5397(8159335) in scavenge on r806u04n03 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5398(8159336) in scavenge on r806u14n03 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5399(8159337) in scavenge on r806u23n01 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5400(8159338) in scavenge on r815u15n06 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5401(8159339) in scavenge on r815u29n03 Nov 04 23:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5402(8159340) in scavenge on r815u29n05 Nov 04 23:16:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_16(8159341) in scavenge on r903u07n03 Nov 04 23:16:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5316(8158844) WEXITSTATUS 1 Nov 04 23:16:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5316(8158844) done Nov 04 23:16:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3583(8159319) WEXITSTATUS 0 Nov 04 23:16:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3583(8159319) done Nov 04 23:16:54 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7569662_1081(7570743) Nov 04 23:16:54 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_520(7572689) Nov 04 23:16:54 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_521(7572690) Nov 04 23:16:54 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_527(7572696) Nov 04 23:16:54 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1937(8125302) Nov 04 23:16:54 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1458(8137918) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:16:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5369(8159179) WEXITSTATUS 1 Nov 04 23:16:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5369(8159179) done Nov 04 23:16:59 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1458(8137918) Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:17:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4925(8158112) WEXITSTATUS 1 Nov 04 23:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4925(8158112) done Nov 04 23:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_24(8158802) WEXITSTATUS 0 Nov 04 23:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_24(8158802) done Nov 04 23:17:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1155(8132733) WEXITSTATUS 0 Nov 04 23:17:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1155(8132733) done Nov 04 23:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4945(8158190) WEXITSTATUS 1 Nov 04 23:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4945(8158190) done Nov 04 23:17:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157544 NodeList=r816u35n03 #CPUs=12 Partition=scavenge Nov 04 23:17:26 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156051 WEXITSTATUS 0 Nov 04 23:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156051 done Nov 04 23:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1944(8151382) WEXITSTATUS 0 Nov 04 23:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1944(8151382) done Nov 04 23:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4518(8155907) WEXITSTATUS 1 Nov 04 23:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4518(8155907) done Nov 04 23:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4424(8155665) WEXITSTATUS 1 Nov 04 23:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4424(8155665) done Nov 04 23:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3584(8159320) WEXITSTATUS 0 Nov 04 23:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3584(8159320) done Nov 04 23:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3555(8158728) WEXITSTATUS 0 Nov 04 23:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3555(8158728) done Nov 04 23:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156050 WEXITSTATUS 0 Nov 04 23:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156050 done Nov 04 23:18:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156054 WEXITSTATUS 0 Nov 04 23:18:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156054 done Nov 04 23:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156056 WEXITSTATUS 0 Nov 04 23:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156056 done Nov 04 23:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_32(8158810) WEXITSTATUS 0 Nov 04 23:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_32(8158810) done Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:19:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5345(8159023) WEXITSTATUS 1 Nov 04 23:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5345(8159023) done Nov 04 23:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3568(8159016) WEXITSTATUS 0 Nov 04 23:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3568(8159016) done Nov 04 23:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3579(8159263) WEXITSTATUS 0 Nov 04 23:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3579(8159263) done Nov 04 23:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_23(8158801) WEXITSTATUS 0 Nov 04 23:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_23(8158801) done Nov 04 23:19:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157545 NodeList=r816u15n09 #CPUs=12 Partition=scavenge Nov 04 23:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_27(8158805) WEXITSTATUS 0 Nov 04 23:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_27(8158805) done Nov 04 23:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5086(8158410) WEXITSTATUS 1 Nov 04 23:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5086(8158410) done Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_333(8159342) in pi_ohern on r917u29n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_334(8159343) in pi_ohern on r917u29n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_335(8159344) in pi_ohern on r917u29n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_336(8159345) in pi_ohern on r917u29n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_337(8159346) in pi_ohern on r917u29n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_338(8159347) in pi_ohern on r917u29n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_339(8159348) in pi_ohern on r917u29n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_340(8159349) in pi_ohern on r917u29n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_341(8159350) in pi_ohern on r917u29n02 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_342(8159351) in pi_ohern on r917u29n02 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_343(8159352) in pi_ohern on r917u29n02 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_344(8159353) in pi_ohern on r917u29n02 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_345(8159354) in pi_ohern on r917u29n02 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_346(8159355) in pi_ohern on r917u29n02 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_347(8159356) in pi_ohern on r917u29n02 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_348(8159357) in pi_ohern on r917u29n02 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_349(8159358) in pi_ohern on r917u29n03 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_350(8159359) in pi_ohern on r917u29n03 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_351(8159360) in pi_ohern on r917u29n03 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_352(8159361) in pi_ohern on r917u29n03 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_353(8159362) in pi_ohern on r917u29n03 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_354(8159363) in pi_ohern on r917u29n03 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_355(8159364) in pi_ohern on r917u29n03 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_356(8159365) in pi_ohern on r917u29n03 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_357(8159366) in pi_ohern on r917u29n04 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_358(8159367) in pi_ohern on r917u29n04 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_359(8159368) in pi_ohern on r917u29n04 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_360(8159369) in pi_ohern on r917u29n04 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_361(8159370) in pi_ohern on r917u29n04 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_362(8159371) in pi_ohern on r917u29n04 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_363(8159372) in pi_ohern on r917u29n04 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_364(8159373) in pi_ohern on r917u29n04 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_365(8159374) in pi_ohern on r815u03n11 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_366(8159375) in pi_ohern on r815u03n12 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_367(8159376) in pi_ohern on r815u03n12 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_368(8159377) in pi_ohern on r815u03n12 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_369(8159378) in pi_ohern on r815u03n12 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3585(8159379) in gpu on r808u11n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3586(8159380) in gpu on r808u14n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3587(8159381) in gpu on r808u17n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3588(8159382) in gpu on r4513u20n01 Nov 04 23:19:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3589(8159383) in gpu on r4513u29n01 Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157546 in scavenge on r815u23n05 Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157547 in scavenge on r815u29n02 Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157656 in scavenge on r815u29n04 Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_787(7572956) WEXITSTATUS 0 Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_787(7572956) done Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1159(7573328) WEXITSTATUS 0 Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1159(7573328) done Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_291(8114557) WEXITSTATUS 0 Nov 04 23:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_291(8114557) done Nov 04 23:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_29(8158807) WEXITSTATUS 0 Nov 04 23:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_29(8158807) done Nov 04 23:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2203(8151995) WEXITSTATUS 0 Nov 04 23:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2203(8151995) done Nov 04 23:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2209(8152001) WEXITSTATUS 0 Nov 04 23:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2209(8152001) done Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5403(8159384) in scavenge on r802u02n03 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5404(8159385) in scavenge on r903u07n01 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5405(8159386) in scavenge on r917u01n04 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5406(8159387) in scavenge on r805u09n01 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5407(8159388) in scavenge on r913u28n04 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5408(8159389) in scavenge on r909u07n03 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5409(8159390) in scavenge on r909u07n04 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5410(8159391) in scavenge on r909u07n04 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5411(8159392) in scavenge on r909u07n04 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5412(8159393) in scavenge on r919u01n03 Nov 04 23:19:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5413(8159394) in scavenge on r919u36n02 Nov 04 23:19:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_17(8159395) in scavenge on r806u06n04 Nov 04 23:20:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4182(8151241) WEXITSTATUS 1 Nov 04 23:20:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4182(8151241) done Nov 04 23:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5366(8159176) WEXITSTATUS 1 Nov 04 23:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5366(8159176) done Nov 04 23:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_31(8158809) WEXITSTATUS 0 Nov 04 23:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_31(8158809) done Nov 04 23:20:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158778 WEXITSTATUS 0 Nov 04 23:20:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158778 done Nov 04 23:20:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1851(8159396) in scavenge on r806u28n01 Nov 04 23:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5375(8159268) WEXITSTATUS 1 Nov 04 23:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5375(8159268) done Nov 04 23:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3589(8159383) WEXITSTATUS 0 Nov 04 23:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3589(8159383) done Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:20:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:20:26 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1461(8138138) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:20:26 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4269(8153751) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:20:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157666 NodeList=r815u15n11 #CPUs=12 Partition=scavenge Nov 04 23:20:28 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4269(8153751) Nov 04 23:20:28 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1461(8138138) Nov 04 23:20:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 23:20:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4619(8156203) WEXITSTATUS 1 Nov 04 23:20:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4619(8156203) done Nov 04 23:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156058 WEXITSTATUS 0 Nov 04 23:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156058 done Nov 04 23:20:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_30(8158808) WEXITSTATUS 0 Nov 04 23:20:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_30(8158808) done Nov 04 23:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4586(8156107) WEXITSTATUS 1 Nov 04 23:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4586(8156107) done Nov 04 23:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156053 WEXITSTATUS 0 Nov 04 23:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156053 done Nov 04 23:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3585(8159379) WEXITSTATUS 0 Nov 04 23:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3585(8159379) done Nov 04 23:20:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:21:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156057 WEXITSTATUS 0 Nov 04 23:21:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156057 done Nov 04 23:21:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_28(8158806) WEXITSTATUS 0 Nov 04 23:21:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_28(8158806) done Nov 04 23:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_34(8158812) WEXITSTATUS 0 Nov 04 23:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_34(8158812) done Nov 04 23:21:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157667 NodeList=r816u03n10 #CPUs=12 Partition=scavenge Nov 04 23:21:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157668 NodeList=r816u35n10 #CPUs=12 Partition=scavenge Nov 04 23:21:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_56(8158834) WEXITSTATUS 0 Nov 04 23:21:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_56(8158834) done Nov 04 23:21:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_53(8158831) WEXITSTATUS 0 Nov 04 23:21:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_53(8158831) done Nov 04 23:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3588(8159382) WEXITSTATUS 0 Nov 04 23:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3588(8159382) done Nov 04 23:21:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156055 WEXITSTATUS 0 Nov 04 23:21:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156055 done Nov 04 23:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_1(8158779) WEXITSTATUS 0 Nov 04 23:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_1(8158779) done Nov 04 23:22:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1923(8151361) WEXITSTATUS 0 Nov 04 23:22:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1923(8151361) done Nov 04 23:22:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156060 WEXITSTATUS 0 Nov 04 23:22:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156060 done Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:22:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_33(8158811) WEXITSTATUS 0 Nov 04 23:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_33(8158811) done Nov 04 23:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_37(8158815) WEXITSTATUS 0 Nov 04 23:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_37(8158815) done Nov 04 23:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 21392: QOSMaxSubmitJobPerUserLimit Nov 04 23:22:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 04 23:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146596_3(8152489) WEXITSTATUS 0 Nov 04 23:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146596_3(8152489) done Nov 04 23:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3586(8159380) WEXITSTATUS 0 Nov 04 23:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3586(8159380) done Nov 04 23:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_52(8158830) WEXITSTATUS 0 Nov 04 23:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_52(8158830) done Nov 04 23:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5350(8159084) WEXITSTATUS 1 Nov 04 23:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5350(8159084) done Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_370(8159397) in pi_ohern on r917u13n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_371(8159398) in pi_ohern on r917u29n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_372(8159399) in pi_ohern on r917u29n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_373(8159400) in pi_ohern on r917u29n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_374(8159401) in pi_ohern on r917u29n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_375(8159402) in pi_ohern on r917u29n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_376(8159403) in pi_ohern on r917u29n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_377(8159404) in pi_ohern on r917u29n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_378(8159405) in pi_ohern on r917u29n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_379(8159406) in pi_ohern on r917u29n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_380(8159407) in pi_ohern on r917u29n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_381(8159408) in pi_ohern on r917u29n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_382(8159409) in pi_ohern on r917u29n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_383(8159410) in pi_ohern on r917u29n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_384(8159411) in pi_ohern on r917u29n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_385(8159412) in pi_ohern on r917u29n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_386(8159413) in pi_ohern on r917u29n02 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_387(8159414) in pi_ohern on r917u29n03 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_388(8159415) in pi_ohern on r917u29n03 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_389(8159416) in pi_ohern on r917u29n03 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_390(8159417) in pi_ohern on r917u29n03 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_391(8159418) in pi_ohern on r917u29n03 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_392(8159419) in pi_ohern on r917u29n03 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_393(8159420) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_394(8159421) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_395(8159422) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_396(8159423) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_397(8159424) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_398(8159425) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_399(8159426) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_400(8159427) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_401(8159428) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_402(8159429) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_403(8159430) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_404(8159431) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_405(8159432) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_406(8159433) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_407(8159434) in pi_ohern on r815u03n11 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_408(8159435) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_409(8159436) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_410(8159437) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_411(8159438) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_412(8159439) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_413(8159440) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_414(8159441) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_415(8159442) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_416(8159443) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_417(8159444) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_418(8159445) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_419(8159446) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_420(8159447) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_421(8159448) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_422(8159449) in pi_ohern on r815u03n12 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_423(8159450) in pi_ohern on r911u32n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_424(8159451) in pi_ohern on r911u32n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_425(8159452) in pi_ohern on r913u32n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_426(8159453) in pi_ohern on r913u32n01 Nov 04 23:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_427(8159454) in pi_ohern on r913u35n01 Nov 04 23:22:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3590(8159455) in gpu on r808u11n01 Nov 04 23:22:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3591(8159456) in gpu on r808u14n01 Nov 04 23:22:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3592(8159457) in gpu on r4513u20n01 Nov 04 23:22:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3593(8159458) in gpu on r4513u29n01 Nov 04 23:22:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157669 in scavenge on r815u29n02 Nov 04 23:22:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157670 in scavenge on r815u35n03 Nov 04 23:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_823(7572992) WEXITSTATUS 0 Nov 04 23:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_823(7572992) done Nov 04 23:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1117(7573286) WEXITSTATUS 0 Nov 04 23:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1117(7573286) done Nov 04 23:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_25(8158803) WEXITSTATUS 0 Nov 04 23:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_25(8158803) done Nov 04 23:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1171(7573340) WEXITSTATUS 0 Nov 04 23:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1171(7573340) done Nov 04 23:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3590(8159455) WEXITSTATUS 0 Nov 04 23:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3590(8159455) done Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5414(8159459) in scavenge on r916u04n01 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5415(8159460) in scavenge on r802u06n04 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5416(8159461) in scavenge on r802u09n03 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5417(8159462) in scavenge on r805u06n03 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5418(8159463) in scavenge on r906u41n01 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5419(8159464) in scavenge on r909u13n03 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5420(8159465) in scavenge on r913u28n02 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5421(8159466) in scavenge on r816u03n05 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5422(8159467) in scavenge on r816u09n05 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5423(8159468) in scavenge on r805u34n01 Nov 04 23:23:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5424(8159469) in scavenge on r805u34n01 Nov 04 23:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_26(8158804) WEXITSTATUS 0 Nov 04 23:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_26(8158804) done Nov 04 23:23:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1296(8149916) WEXITSTATUS 0 Nov 04 23:23:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1296(8149916) done Nov 04 23:23:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1961(8125371) WEXITSTATUS 1 Nov 04 23:23:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1961(8125371) done Nov 04 23:23:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3592(8159457) WEXITSTATUS 0 Nov 04 23:23:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3592(8159457) done Nov 04 23:23:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3445(8135293) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:23:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1460(8138137) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:23:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1462(8138139) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:23:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4762(8157333) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:23:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_50(8158828) WEXITSTATUS 0 Nov 04 23:23:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_50(8158828) done Nov 04 23:23:32 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3445(8135293) Nov 04 23:23:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1460(8138137) Nov 04 23:23:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1462(8138139) Nov 04 23:23:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4762(8157333) Nov 04 23:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5424(8159469) WEXITSTATUS 1 Nov 04 23:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5424(8159469) done Nov 04 23:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_48(8158826) WEXITSTATUS 0 Nov 04 23:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_48(8158826) done Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5420(8159465) WEXITSTATUS 1 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5420(8159465) done Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:23:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5399(8159337) WEXITSTATUS 1 Nov 04 23:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5399(8159337) done Nov 04 23:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_46(8158824) WEXITSTATUS 0 Nov 04 23:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_46(8158824) done Nov 04 23:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_39(8158817) WEXITSTATUS 0 Nov 04 23:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_39(8158817) done Nov 04 23:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_38(8158816) WEXITSTATUS 0 Nov 04 23:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_38(8158816) done Nov 04 23:24:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5002(8158286) WEXITSTATUS 1 Nov 04 23:24:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5002(8158286) done Nov 04 23:24:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_12(8158790) WEXITSTATUS 0 Nov 04 23:24:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_12(8158790) done Nov 04 23:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_22(8158800) WEXITSTATUS 0 Nov 04 23:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_22(8158800) done Nov 04 23:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_21(8158799) WEXITSTATUS 0 Nov 04 23:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_21(8158799) done Nov 04 23:24:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_3(8158781) WEXITSTATUS 0 Nov 04 23:24:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_3(8158781) done Nov 04 23:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_2(8158780) WEXITSTATUS 0 Nov 04 23:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_2(8158780) done Nov 04 23:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_55(8158833) WEXITSTATUS 0 Nov 04 23:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_55(8158833) done Nov 04 23:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_45(8158823) WEXITSTATUS 0 Nov 04 23:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_45(8158823) done Nov 04 23:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5226(8158613) WEXITSTATUS 1 Nov 04 23:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5226(8158613) done Nov 04 23:24:28 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1947(8125337) Nov 04 23:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5324(8158897) WEXITSTATUS 1 Nov 04 23:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5324(8158897) done Nov 04 23:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_49(8158827) WEXITSTATUS 0 Nov 04 23:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_49(8158827) done Nov 04 23:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_47(8158825) WEXITSTATUS 0 Nov 04 23:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_47(8158825) done Nov 04 23:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_40(8158818) WEXITSTATUS 0 Nov 04 23:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_40(8158818) done Nov 04 23:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_36(8158814) WEXITSTATUS 0 Nov 04 23:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_36(8158814) done Nov 04 23:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5116(8158449) WEXITSTATUS 1 Nov 04 23:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5116(8158449) done Nov 04 23:24:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_75(8158878) WEXITSTATUS 0 Nov 04 23:24:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_75(8158878) done Nov 04 23:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_82(8158885) WEXITSTATUS 0 Nov 04 23:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_82(8158885) done Nov 04 23:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_35(8158813) WEXITSTATUS 0 Nov 04 23:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_35(8158813) done Nov 04 23:24:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3591(8159456) WEXITSTATUS 0 Nov 04 23:24:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3591(8159456) done Nov 04 23:24:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8128083 Nov 04 23:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5207(8158570) WEXITSTATUS 1 Nov 04 23:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5207(8158570) done Nov 04 23:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156064 WEXITSTATUS 0 Nov 04 23:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156064 done Nov 04 23:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3582(8159318) WEXITSTATUS 0 Nov 04 23:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3582(8159318) done Nov 04 23:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_4(8158782) WEXITSTATUS 0 Nov 04 23:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_4(8158782) done Nov 04 23:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_54(8158832) WEXITSTATUS 0 Nov 04 23:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_54(8158832) done Nov 04 23:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_16(8158794) WEXITSTATUS 0 Nov 04 23:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_16(8158794) done Nov 04 23:25:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8139112 NodeList=r815u35n07 #CPUs=8 Partition=gpu_devel Nov 04 23:25:28 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8152263 Nov 04 23:25:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156059 WEXITSTATUS 0 Nov 04 23:25:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156059 done Nov 04 23:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_8(8158786) WEXITSTATUS 0 Nov 04 23:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_8(8158786) done Nov 04 23:25:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_57(8158835) WEXITSTATUS 0 Nov 04 23:25:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_57(8158835) done Nov 04 23:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5396(8159334) WEXITSTATUS 1 Nov 04 23:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5396(8159334) done Nov 04 23:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_83(8158886) WEXITSTATUS 0 Nov 04 23:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_83(8158886) done Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4826(8157805) WEXITSTATUS 1 Nov 04 23:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4826(8157805) done Nov 04 23:25:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_6(8158784) WEXITSTATUS 0 Nov 04 23:25:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_6(8158784) done Nov 04 23:25:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_58(8158836) WEXITSTATUS 0 Nov 04 23:25:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_58(8158836) done Nov 04 23:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156061 WEXITSTATUS 0 Nov 04 23:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156061 done Nov 04 23:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_44(8158822) WEXITSTATUS 0 Nov 04 23:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_44(8158822) done Nov 04 23:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_7(8158785) WEXITSTATUS 0 Nov 04 23:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_7(8158785) done Nov 04 23:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5306(8158768) WEXITSTATUS 1 Nov 04 23:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5306(8158768) done Nov 04 23:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4941(8158186) WEXITSTATUS 1 Nov 04 23:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4941(8158186) done Nov 04 23:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4522(8155923) WEXITSTATUS 1 Nov 04 23:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4522(8155923) done Nov 04 23:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2306(8152223) WEXITSTATUS 0 Nov 04 23:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2306(8152223) done Nov 04 23:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_51(8158829) WEXITSTATUS 0 Nov 04 23:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_51(8158829) done Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_428(8159470) in pi_ohern on r917u13n02 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_429(8159471) in pi_ohern on r917u13n02 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_430(8159472) in pi_ohern on r917u13n02 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_431(8159473) in pi_ohern on r917u13n02 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_432(8159474) in pi_ohern on r917u13n02 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_433(8159475) in pi_ohern on r917u13n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_434(8159476) in pi_ohern on r917u13n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_435(8159477) in pi_ohern on r917u13n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_436(8159478) in pi_ohern on r917u13n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_437(8159479) in pi_ohern on r917u13n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_438(8159480) in pi_ohern on r917u13n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_439(8159481) in pi_ohern on r917u13n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_440(8159482) in pi_ohern on r917u13n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_441(8159483) in pi_ohern on r917u13n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_442(8159484) in pi_ohern on r917u13n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_443(8159485) in pi_ohern on r917u13n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_444(8159486) in pi_ohern on r917u13n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_445(8159487) in pi_ohern on r917u13n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_446(8159488) in pi_ohern on r917u13n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_447(8159489) in pi_ohern on r917u13n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_448(8159490) in pi_ohern on r917u13n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_449(8159491) in pi_ohern on r917u29n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_450(8159492) in pi_ohern on r917u29n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_451(8159493) in pi_ohern on r917u29n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_452(8159494) in pi_ohern on r917u29n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_453(8159495) in pi_ohern on r917u29n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_454(8159496) in pi_ohern on r917u29n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_455(8159497) in pi_ohern on r917u29n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_456(8159498) in pi_ohern on r917u29n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_457(8159499) in pi_ohern on r917u29n02 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_458(8159500) in pi_ohern on r917u29n02 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_459(8159501) in pi_ohern on r917u29n02 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_460(8159502) in pi_ohern on r917u29n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_461(8159503) in pi_ohern on r917u29n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_462(8159504) in pi_ohern on r917u29n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_463(8159505) in pi_ohern on r917u29n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_464(8159506) in pi_ohern on r917u29n03 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_465(8159507) in pi_ohern on r917u29n04 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_466(8159508) in pi_ohern on r815u03n11 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_467(8159509) in pi_ohern on r815u03n11 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_468(8159510) in pi_ohern on r815u03n11 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_469(8159511) in pi_ohern on r815u03n11 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_470(8159512) in pi_ohern on r815u03n11 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_471(8159513) in pi_ohern on r815u03n11 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_472(8159514) in pi_ohern on r815u03n11 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_473(8159515) in pi_ohern on r815u03n11 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_474(8159516) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_475(8159517) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_476(8159518) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_477(8159519) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_478(8159520) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_479(8159521) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_480(8159522) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_481(8159523) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_482(8159524) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_483(8159525) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_484(8159526) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_485(8159527) in pi_ohern on r815u03n12 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_486(8159528) in pi_ohern on r911u32n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_487(8159529) in pi_ohern on r911u32n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_488(8159530) in pi_ohern on r807u22n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_489(8159531) in pi_ohern on r807u24n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_490(8159532) in pi_ohern on r807u24n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_491(8159533) in pi_ohern on r903u12n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_492(8159534) in pi_ohern on r903u12n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_493(8159535) in pi_ohern on r903u12n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_494(8159536) in pi_ohern on r903u26n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_495(8159537) in pi_ohern on r903u26n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_496(8159538) in pi_ohern on r903u26n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5219(8158606) WEXITSTATUS 1 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5219(8158606) done Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:139 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_78(8158881) WEXITSTATUS 0 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_78(8158881) done Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_497(8159539) in pi_ohern on r913u32n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_498(8159540) in pi_ohern on r913u32n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_499(8159541) in pi_ohern on r913u35n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_500(8159542) in pi_ohern on r913u35n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_501(8159543) in pi_ohern on r913u35n01 Nov 04 23:26:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_502(8159544) in pi_ohern on r913u35n01 Nov 04 23:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3594(8159545) in gpu on r808u11n01 Nov 04 23:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3595(8159546) in gpu on r808u14n01 Nov 04 23:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3596(8159547) in gpu on r808u14n01 Nov 04 23:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3597(8159548) in gpu on r4513u20n01 Nov 04 23:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 23:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157671 in scavenge on r815u15n11 Nov 04 23:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157699 in scavenge on r815u23n06 Nov 04 23:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157700 in scavenge on r815u35n03 Nov 04 23:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1089(7573258) WEXITSTATUS 0 Nov 04 23:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1089(7573258) done Nov 04 23:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_420(8114687) WEXITSTATUS 0 Nov 04 23:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_420(8114687) done Nov 04 23:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1154(8132732) WEXITSTATUS 0 Nov 04 23:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1154(8132732) done Nov 04 23:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_5(8158783) WEXITSTATUS 0 Nov 04 23:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_5(8158783) done Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5425(8159549) in scavenge on r801u04n03 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5426(8159550) in scavenge on r801u06n03 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5427(8159551) in scavenge on r802u04n03 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5428(8159552) in scavenge on r802u06n03 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5429(8159553) in scavenge on r802u30n01 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5430(8159554) in scavenge on r802u32n04 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5431(8159555) in scavenge on r806u02n02 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5432(8159556) in scavenge on r806u23n01 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5433(8159557) in scavenge on r913u24n04 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5434(8159558) in scavenge on r913u28n02 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5435(8159559) in scavenge on r918u01n03 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5436(8159560) in scavenge on r919u36n02 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5437(8159561) in scavenge on r816u35n06 Nov 04 23:26:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5438(8159562) in scavenge on r805u34n01 Nov 04 23:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_18(8158796) WEXITSTATUS 0 Nov 04 23:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_18(8158796) done Nov 04 23:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_41(8158819) WEXITSTATUS 0 Nov 04 23:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_41(8158819) done Nov 04 23:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156063 WEXITSTATUS 0 Nov 04 23:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156063 done Nov 04 23:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_13(8158791) WEXITSTATUS 0 Nov 04 23:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_13(8158791) done Nov 04 23:26:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159563 InitPrio=61739 usec=7390 Nov 04 23:26:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159563 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 23:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_73(8158876) WEXITSTATUS 0 Nov 04 23:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_73(8158876) done Nov 04 23:27:00 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2054(8125841) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:27:00 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1402(8135935) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:27:00 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1463(8138140) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3595(8159546) WEXITSTATUS 0 Nov 04 23:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3595(8159546) done Nov 04 23:27:01 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2054(8125841) Nov 04 23:27:02 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1402(8135935) Nov 04 23:27:02 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1463(8138140) Nov 04 23:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_89(8158892) WEXITSTATUS 0 Nov 04 23:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_89(8158892) done Nov 04 23:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_80(8158883) WEXITSTATUS 0 Nov 04 23:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_80(8158883) done Nov 04 23:27:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_87(8158890) WEXITSTATUS 0 Nov 04 23:27:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_87(8158890) done Nov 04 23:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_43(8158821) WEXITSTATUS 0 Nov 04 23:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_43(8158821) done Nov 04 23:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4956(8158206) WEXITSTATUS 1 Nov 04 23:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4956(8158206) done Nov 04 23:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_72(8158875) WEXITSTATUS 0 Nov 04 23:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_72(8158875) done Nov 04 23:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3535(8158455) WEXITSTATUS 0 Nov 04 23:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3535(8158455) done Nov 04 23:27:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1951(8125348) Nov 04 23:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_117(8158933) WEXITSTATUS 0 Nov 04 23:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_117(8158933) done Nov 04 23:27:32 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_81(8158884) WEXITSTATUS 0 Nov 04 23:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_81(8158884) done Nov 04 23:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3587(8159381) WEXITSTATUS 0 Nov 04 23:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3587(8159381) done Nov 04 23:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_74(8158877) WEXITSTATUS 0 Nov 04 23:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_74(8158877) done Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:27:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_79(8158882) WEXITSTATUS 0 Nov 04 23:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_79(8158882) done Nov 04 23:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5193(8158556) WEXITSTATUS 1 Nov 04 23:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5193(8158556) done Nov 04 23:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_14(8158792) WEXITSTATUS 0 Nov 04 23:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_14(8158792) done Nov 04 23:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_15(8158793) WEXITSTATUS 0 Nov 04 23:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_15(8158793) done Nov 04 23:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5258(8158692) WEXITSTATUS 1 Nov 04 23:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5258(8158692) done Nov 04 23:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_116(8158932) WEXITSTATUS 0 Nov 04 23:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_116(8158932) done Nov 04 23:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5377(8159270) WEXITSTATUS 1 Nov 04 23:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5377(8159270) done Nov 04 23:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_42(8158820) WEXITSTATUS 0 Nov 04 23:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_42(8158820) done Nov 04 23:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_88(8158891) WEXITSTATUS 0 Nov 04 23:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_88(8158891) done Nov 04 23:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_61(8158864) WEXITSTATUS 0 Nov 04 23:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_61(8158864) done Nov 04 23:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_10(8158788) WEXITSTATUS 0 Nov 04 23:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_10(8158788) done Nov 04 23:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_19(8158797) WEXITSTATUS 0 Nov 04 23:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_19(8158797) done Nov 04 23:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_86(8158889) WEXITSTATUS 0 Nov 04 23:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_86(8158889) done Nov 04 23:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_20(8158798) WEXITSTATUS 0 Nov 04 23:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_20(8158798) done Nov 04 23:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2254(8152171) WEXITSTATUS 0 Nov 04 23:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2254(8152171) done Nov 04 23:28:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_70(8158873) WEXITSTATUS 0 Nov 04 23:28:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_70(8158873) done Nov 04 23:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_11(8158789) WEXITSTATUS 0 Nov 04 23:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_11(8158789) done Nov 04 23:28:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5140(8158487) WEXITSTATUS 1 Nov 04 23:28:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5140(8158487) done Nov 04 23:28:26 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 23:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2212(8152004) WEXITSTATUS 0 Nov 04 23:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2212(8152004) done Nov 04 23:28:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8135483 Nov 04 23:28:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8135483 error Job/step already completing or completed Nov 04 23:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1761(8150972) WEXITSTATUS 0 Nov 04 23:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1761(8150972) done Nov 04 23:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156065 WEXITSTATUS 0 Nov 04 23:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156065 done Nov 04 23:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_76(8158879) WEXITSTATUS 0 Nov 04 23:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_76(8158879) done Nov 04 23:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_63(8158866) WEXITSTATUS 0 Nov 04 23:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_63(8158866) done Nov 04 23:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_9(8158787) WEXITSTATUS 0 Nov 04 23:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_9(8158787) done Nov 04 23:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_130(8158946) WEXITSTATUS 0 Nov 04 23:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_130(8158946) done Nov 04 23:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_110(8158926) WEXITSTATUS 0 Nov 04 23:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_110(8158926) done Nov 04 23:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_104(8158920) WEXITSTATUS 0 Nov 04 23:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_104(8158920) done Nov 04 23:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_103(8158919) WEXITSTATUS 0 Nov 04 23:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_103(8158919) done Nov 04 23:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_112(8158928) WEXITSTATUS 0 Nov 04 23:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_112(8158928) done Nov 04 23:28:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_84(8158887) WEXITSTATUS 0 Nov 04 23:28:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_84(8158887) done Nov 04 23:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_133(8158972) WEXITSTATUS 0 Nov 04 23:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_133(8158972) done Nov 04 23:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_77(8158880) WEXITSTATUS 0 Nov 04 23:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_77(8158880) done Nov 04 23:28:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_64(8158867) WEXITSTATUS 0 Nov 04 23:28:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_64(8158867) done Nov 04 23:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_105(8158921) WEXITSTATUS 0 Nov 04 23:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_105(8158921) done Nov 04 23:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156069 WEXITSTATUS 0 Nov 04 23:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156069 done Nov 04 23:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4784(8157659) WEXITSTATUS 1 Nov 04 23:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4784(8157659) done Nov 04 23:29:11 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:2100 Nov 04 23:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_113(8158929) WEXITSTATUS 0 Nov 04 23:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_113(8158929) done Nov 04 23:29:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1856(8151182) WEXITSTATUS 0 Nov 04 23:29:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1856(8151182) done Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_503(8159564) in pi_ohern on r917u13n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_504(8159565) in pi_ohern on r917u13n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_505(8159566) in pi_ohern on r917u13n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_506(8159567) in pi_ohern on r917u13n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_507(8159568) in pi_ohern on r917u13n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_508(8159569) in pi_ohern on r917u13n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_509(8159570) in pi_ohern on r917u13n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_510(8159571) in pi_ohern on r917u13n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_511(8159572) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_512(8159573) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_513(8159574) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_514(8159575) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_515(8159576) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_516(8159577) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_517(8159578) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_518(8159579) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_519(8159580) in pi_ohern on r917u13n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_520(8159581) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_521(8159582) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_522(8159583) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_523(8159584) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_524(8159585) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_525(8159586) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_526(8159587) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_527(8159588) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_528(8159589) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_529(8159590) in pi_ohern on r917u13n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_530(8159591) in pi_ohern on r917u29n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_531(8159592) in pi_ohern on r917u29n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_532(8159593) in pi_ohern on r917u29n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_533(8159594) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_534(8159595) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_535(8159596) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_536(8159597) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_537(8159598) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_538(8159599) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_539(8159600) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_540(8159601) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_541(8159602) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_542(8159603) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_543(8159604) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_544(8159605) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_545(8159606) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_546(8159607) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_547(8159608) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_548(8159609) in pi_ohern on r917u29n02 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_549(8159610) in pi_ohern on r917u29n03 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_550(8159611) in pi_ohern on r917u29n03 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_551(8159612) in pi_ohern on r917u29n03 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_552(8159613) in pi_ohern on r917u29n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_553(8159614) in pi_ohern on r917u29n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_554(8159615) in pi_ohern on r917u29n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_555(8159616) in pi_ohern on r917u29n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_556(8159617) in pi_ohern on r917u29n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_557(8159618) in pi_ohern on r917u29n04 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_558(8159619) in pi_ohern on r815u03n11 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_559(8159620) in pi_ohern on r815u03n12 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_560(8159621) in pi_ohern on r807u24n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_561(8159622) in pi_ohern on r807u24n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_562(8159623) in pi_ohern on r807u24n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_563(8159624) in pi_ohern on r807u24n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_564(8159625) in pi_ohern on r807u24n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_565(8159626) in pi_ohern on r807u24n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_566(8159627) in pi_ohern on r903u12n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_567(8159628) in pi_ohern on r903u12n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_568(8159629) in pi_ohern on r903u12n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_569(8159630) in pi_ohern on r903u12n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_570(8159631) in pi_ohern on r903u26n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_571(8159632) in pi_ohern on r903u26n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_572(8159633) in pi_ohern on r903u26n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_573(8159634) in pi_ohern on r903u26n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_574(8159635) in pi_ohern on r903u26n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_575(8159636) in pi_ohern on r903u26n01 Nov 04 23:29:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_576(8159637) in pi_ohern on r903u26n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_577(8159638) in pi_ohern on r903u26n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_578(8159639) in pi_ohern on r903u26n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_579(8159640) in pi_ohern on r903u26n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_580(8159641) in pi_ohern on r903u26n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_581(8159642) in pi_ohern on r903u26n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_582(8159643) in pi_ohern on r913u32n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_583(8159644) in pi_ohern on r913u32n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_584(8159645) in pi_ohern on r913u35n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_585(8159646) in pi_ohern on r913u35n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_586(8159647) in pi_ohern on r913u35n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_587(8159648) in pi_ohern on r913u35n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_588(8159649) in pi_ohern on r913u35n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_589(8159650) in pi_ohern on r913u35n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3598(8159651) in gpu on r808u14n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3599(8159652) in gpu on r808u17n01 Nov 04 23:29:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3600(8159653) in gpu on r4513u23n01 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157701 in scavenge on r815u03n06 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157702 in scavenge on r815u09n04 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_62(8158865) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_62(8158865) done Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1149(7573318) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1149(7573318) done Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_575(8159636) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_575(8159636) done Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_579(8159640) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_579(8159640) done Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_581(8159642) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_581(8159642) done Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_576(8159637) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_576(8159637) done Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_580(8159641) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_580(8159641) done Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_577(8159638) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_577(8159638) done Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_578(8159639) WEXITSTATUS 0 Nov 04 23:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_578(8159639) done Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_587(8159648) WEXITSTATUS 0 Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_587(8159648) done Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_585(8159646) WEXITSTATUS 0 Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_585(8159646) done Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_589(8159650) WEXITSTATUS 0 Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_589(8159650) done Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_588(8159649) WEXITSTATUS 0 Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_588(8159649) done Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_586(8159647) WEXITSTATUS 0 Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_586(8159647) done Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_584(8159645) WEXITSTATUS 0 Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_584(8159645) done Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_582(8159643) WEXITSTATUS 0 Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_582(8159643) done Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_583(8159644) WEXITSTATUS 0 Nov 04 23:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_583(8159644) done Nov 04 23:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3655(8137791) WEXITSTATUS 1 Nov 04 23:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3655(8137791) done Nov 04 23:29:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5439(8159654) in scavenge on r908u28n02 Nov 04 23:29:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5440(8159655) in scavenge on r801u04n03 Nov 04 23:29:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5441(8159656) in scavenge on r816u23n10 Nov 04 23:29:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5442(8159657) in scavenge on r806u04n01 Nov 04 23:29:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5443(8159658) in scavenge on r904u04n01 Nov 04 23:29:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5444(8159659) in scavenge on r909u13n01 Nov 04 23:29:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5445(8159660) in scavenge on r909u13n02 Nov 04 23:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5174(8158528) WEXITSTATUS 1 Nov 04 23:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5174(8158528) done Nov 04 23:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_137(8158976) WEXITSTATUS 0 Nov 04 23:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_137(8158976) done Nov 04 23:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_135(8158974) WEXITSTATUS 0 Nov 04 23:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_135(8158974) done Nov 04 23:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_240(8159139) WEXITSTATUS 0 Nov 04 23:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_240(8159139) done Nov 04 23:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_59(8158862) WEXITSTATUS 0 Nov 04 23:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_59(8158862) done Nov 04 23:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5059(8158364) WEXITSTATUS 1 Nov 04 23:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5059(8158364) done Nov 04 23:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156066 WEXITSTATUS 0 Nov 04 23:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156066 done Nov 04 23:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_111(8158927) WEXITSTATUS 0 Nov 04 23:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_111(8158927) done Nov 04 23:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3597(8159548) WEXITSTATUS 0 Nov 04 23:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3597(8159548) done Nov 04 23:30:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1852(8159661) in scavenge on r806u28n01 Nov 04 23:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_65(8158868) WEXITSTATUS 0 Nov 04 23:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_65(8158868) done Nov 04 23:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2859(8128504) WEXITSTATUS 1 Nov 04 23:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2859(8128504) done Nov 04 23:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_115(8158931) WEXITSTATUS 0 Nov 04 23:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_115(8158931) done Nov 04 23:30:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_532(7572701) Nov 04 23:30:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1957(8125360) Nov 04 23:30:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1989(8125568) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:30:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2403(8126438) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:30:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1403(8135936) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:30:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4446(8155759) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:30:05 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2403(8126438) Nov 04 23:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_109(8158925) WEXITSTATUS 0 Nov 04 23:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_109(8158925) done Nov 04 23:30:05 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4446(8155759) Nov 04 23:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_132(8158971) WEXITSTATUS 0 Nov 04 23:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_132(8158971) done Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1989(8125568) Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1403(8135936) Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:30:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_97(8158913) WEXITSTATUS 0 Nov 04 23:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_97(8158913) done Nov 04 23:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_123(8158939) WEXITSTATUS 0 Nov 04 23:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_123(8158939) done Nov 04 23:30:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8159662 NodeList=(null) usec=39694 Nov 04 23:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_134(8158973) WEXITSTATUS 0 Nov 04 23:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_134(8158973) done Nov 04 23:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_129(8158945) WEXITSTATUS 0 Nov 04 23:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_129(8158945) done Nov 04 23:30:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159662 NodeList=r801u06n01 #CPUs=1 Partition=education Nov 04 23:30:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321883 NodeList=r4512u32n01 #CPUs=1 Partition=transfer Nov 04 23:30:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157703 NodeList=r816u23n08 #CPUs=12 Partition=scavenge Nov 04 23:30:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5388(8159326) WEXITSTATUS 1 Nov 04 23:30:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5388(8159326) done Nov 04 23:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156062 WEXITSTATUS 0 Nov 04 23:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156062 done Nov 04 23:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_90(8158893) WEXITSTATUS 0 Nov 04 23:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_90(8158893) done Nov 04 23:30:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_114(8158930) WEXITSTATUS 0 Nov 04 23:30:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_114(8158930) done Nov 04 23:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_108(8158924) WEXITSTATUS 0 Nov 04 23:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_108(8158924) done Nov 04 23:30:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4576(8156089) WEXITSTATUS 1 Nov 04 23:30:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4576(8156089) done Nov 04 23:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5425(8159549) WEXITSTATUS 1 Nov 04 23:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5425(8159549) done Nov 04 23:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_94(8158910) WEXITSTATUS 0 Nov 04 23:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_94(8158910) done Nov 04 23:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_207(8159063) WEXITSTATUS 0 Nov 04 23:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_207(8159063) done Nov 04 23:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_96(8158912) WEXITSTATUS 0 Nov 04 23:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_96(8158912) done Nov 04 23:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_100(8158916) WEXITSTATUS 0 Nov 04 23:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_100(8158916) done Nov 04 23:31:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_69(8158872) WEXITSTATUS 0 Nov 04 23:31:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_69(8158872) done Nov 04 23:31:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_126(8158942) WEXITSTATUS 0 Nov 04 23:31:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_126(8158942) done Nov 04 23:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135538 WEXITSTATUS 0 Nov 04 23:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135538 done Nov 04 23:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321883 WEXITSTATUS 0 Nov 04 23:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321883 done Nov 04 23:31:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321883 Nov 04 23:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2174(8151966) WEXITSTATUS 0 Nov 04 23:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2174(8151966) done Nov 04 23:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_67(8158870) WEXITSTATUS 0 Nov 04 23:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_67(8158870) done Nov 04 23:31:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4999(8158283) WEXITSTATUS 1 Nov 04 23:31:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4999(8158283) done Nov 04 23:31:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156068 WEXITSTATUS 0 Nov 04 23:31:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156068 done Nov 04 23:31:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157704 NodeList=r816u35n01 #CPUs=12 Partition=scavenge Nov 04 23:31:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2264(8152181) WEXITSTATUS 0 Nov 04 23:31:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2264(8152181) done Nov 04 23:31:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_169(8159008) WEXITSTATUS 0 Nov 04 23:31:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_169(8159008) done Nov 04 23:31:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_120(8158936) WEXITSTATUS 0 Nov 04 23:31:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_120(8158936) done Nov 04 23:31:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_107(8158923) WEXITSTATUS 0 Nov 04 23:31:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_107(8158923) done Nov 04 23:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_152(8158991) WEXITSTATUS 0 Nov 04 23:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_152(8158991) done Nov 04 23:31:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_121(8158937) WEXITSTATUS 0 Nov 04 23:31:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_121(8158937) done Nov 04 23:31:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5064(8158369) WEXITSTATUS 1 Nov 04 23:31:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5064(8158369) done Nov 04 23:31:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_131(8158947) WEXITSTATUS 0 Nov 04 23:31:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_131(8158947) done Nov 04 23:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_17(8158795) WEXITSTATUS 0 Nov 04 23:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_17(8158795) done Nov 04 23:31:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5192(8158555) WEXITSTATUS 1 Nov 04 23:31:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5192(8158555) done Nov 04 23:31:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3545(8158626) WEXITSTATUS 0 Nov 04 23:31:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3545(8158626) done Nov 04 23:31:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_1(8158852) WEXITSTATUS 0 Nov 04 23:31:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_1(8158852) done Nov 04 23:31:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_85(8158888) WEXITSTATUS 0 Nov 04 23:31:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_85(8158888) done Nov 04 23:31:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156067 WEXITSTATUS 0 Nov 04 23:31:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156067 done Nov 04 23:31:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4309(8154075) WEXITSTATUS 1 Nov 04 23:31:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4309(8154075) done Nov 04 23:31:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_118(8158934) WEXITSTATUS 0 Nov 04 23:31:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_118(8158934) done Nov 04 23:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159663 InitPrio=17454 usec=3843 Nov 04 23:32:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_95(8158911) WEXITSTATUS 0 Nov 04 23:32:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_95(8158911) done Nov 04 23:32:01 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159663 NodeList=r801u11n02 #CPUs=4 Partition=pi_esi Nov 04 23:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_71(8158874) WEXITSTATUS 0 Nov 04 23:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_71(8158874) done Nov 04 23:32:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159664 InitPrio=17454 usec=3611 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:32:07 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:32:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159664 NodeList=r801u23n02 #CPUs=4 Partition=pi_esi Nov 04 23:32:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_127(8158943) WEXITSTATUS 0 Nov 04 23:32:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_127(8158943) done Nov 04 23:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158775 WEXITSTATUS 0 Nov 04 23:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158775 done Nov 04 23:32:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156070 WEXITSTATUS 0 Nov 04 23:32:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156070 done Nov 04 23:32:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159665 InitPrio=17454 usec=4255 Nov 04 23:32:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159665 NodeList=r801u23n02 #CPUs=4 Partition=pi_esi Nov 04 23:32:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_153(8158992) WEXITSTATUS 0 Nov 04 23:32:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_153(8158992) done Nov 04 23:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_136(8158975) WEXITSTATUS 0 Nov 04 23:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_136(8158975) done Nov 04 23:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_139(8158978) WEXITSTATUS 0 Nov 04 23:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_139(8158978) done Nov 04 23:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1254(8135156) WEXITSTATUS 0 Nov 04 23:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1254(8135156) done Nov 04 23:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_60(8158863) WEXITSTATUS 0 Nov 04 23:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_60(8158863) done Nov 04 23:32:30 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_560(7572729) Nov 04 23:32:30 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_650(7572819) Nov 04 23:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_91(8158907) WEXITSTATUS 0 Nov 04 23:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_91(8158907) done Nov 04 23:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159663 WEXITSTATUS 0 Nov 04 23:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159663 done Nov 04 23:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4835(8157832) WEXITSTATUS 1 Nov 04 23:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4835(8157832) done Nov 04 23:32:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5171(8158525) WEXITSTATUS 1 Nov 04 23:32:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5171(8158525) done Nov 04 23:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159665 WEXITSTATUS 0 Nov 04 23:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159665 done Nov 04 23:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159664 WEXITSTATUS 0 Nov 04 23:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159664 done Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_590(8159666) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_591(8159667) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_592(8159668) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_593(8159669) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_594(8159670) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_595(8159671) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_596(8159672) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_597(8159673) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_598(8159674) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_599(8159675) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_600(8159676) in pi_ohern on r917u13n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_601(8159677) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_602(8159678) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_603(8159679) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_604(8159680) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_605(8159681) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_606(8159682) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_607(8159683) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_608(8159684) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_609(8159685) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_610(8159686) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_611(8159687) in pi_ohern on r917u13n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_612(8159688) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_613(8159689) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_614(8159690) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_615(8159691) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_616(8159692) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_617(8159693) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_618(8159694) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_619(8159695) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_620(8159696) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_621(8159697) in pi_ohern on r917u13n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_622(8159698) in pi_ohern on r917u29n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_623(8159699) in pi_ohern on r917u29n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_624(8159700) in pi_ohern on r917u29n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_625(8159701) in pi_ohern on r917u29n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_626(8159702) in pi_ohern on r917u29n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_627(8159703) in pi_ohern on r917u29n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_628(8159704) in pi_ohern on r917u29n02 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_629(8159705) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_630(8159706) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_631(8159707) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_632(8159708) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_633(8159709) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_634(8159710) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_635(8159711) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_636(8159712) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_637(8159713) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_638(8159714) in pi_ohern on r917u29n03 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_639(8159715) in pi_ohern on r917u29n04 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_640(8159716) in pi_ohern on r815u03n11 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_641(8159717) in pi_ohern on r815u03n11 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_642(8159718) in pi_ohern on r815u03n12 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_643(8159719) in pi_ohern on r815u03n12 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_644(8159720) in pi_ohern on r815u03n12 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_645(8159721) in pi_ohern on r815u03n12 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_646(8159722) in pi_ohern on r815u03n12 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_647(8159723) in pi_ohern on r815u03n12 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_648(8159724) in pi_ohern on r807u24n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_649(8159725) in pi_ohern on r807u24n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_650(8159726) in pi_ohern on r807u24n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_651(8159727) in pi_ohern on r807u24n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_652(8159728) in pi_ohern on r807u24n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_653(8159729) in pi_ohern on r807u24n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_654(8159730) in pi_ohern on r807u24n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_655(8159731) in pi_ohern on r913u32n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:132 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_151(8158990) WEXITSTATUS 0 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_151(8158990) done Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_656(8159732) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_657(8159733) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_658(8159734) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_659(8159735) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_660(8159736) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_661(8159737) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_662(8159738) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_663(8159739) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_664(8159740) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_665(8159741) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_666(8159742) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_667(8159743) in pi_ohern on r913u35n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158777_668(8158777) in pi_ohern on r903u12n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_1(8159744) in pi_ohern on r903u12n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_2(8159745) in pi_ohern on r903u24n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_3(8159746) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_4(8159747) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_5(8159748) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_6(8159749) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_7(8159750) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_8(8159751) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_9(8159752) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_10(8159753) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_11(8159754) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_12(8159755) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_13(8159756) in pi_ohern on r903u26n01 Nov 04 23:32:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_14(8159757) in pi_ohern on r903u26n01 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n03 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3601(8159758) in gpu on r808u11n01 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3602(8159759) in gpu on r4513u20n01 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1853(8159760) in day on r815u03n02 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1854(8159761) in day on r815u03n02 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157705 in scavenge on r815u23n01 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157706 in scavenge on r815u03n02 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157707 in scavenge on r815u03n02 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157708 in scavenge on r815u09n04 Nov 04 23:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157709 in scavenge on r815u09n08 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_619(8159695) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_619(8159695) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_617(8159693) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_617(8159693) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_615(8159691) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_615(8159691) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_597(8159673) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_597(8159673) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_591(8159667) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_591(8159667) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_620(8159696) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_620(8159696) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_618(8159694) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_618(8159694) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_599(8159675) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_599(8159675) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_613(8159689) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_613(8159689) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_614(8159690) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_614(8159690) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_595(8159671) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_595(8159671) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_590(8159666) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_590(8159666) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_616(8159692) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_616(8159692) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_621(8159697) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_621(8159697) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_593(8159669) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_593(8159669) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_612(8159688) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_612(8159688) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_596(8159672) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_596(8159672) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_592(8159668) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_592(8159668) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_600(8159676) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_600(8159676) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_598(8159674) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_598(8159674) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_594(8159670) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_594(8159670) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_606(8159682) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_606(8159682) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_607(8159683) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_607(8159683) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_609(8159685) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_609(8159685) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_608(8159684) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_608(8159684) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_610(8159686) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_610(8159686) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_602(8159678) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_602(8159678) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_601(8159677) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_601(8159677) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_604(8159680) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_604(8159680) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_603(8159679) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_603(8159679) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_605(8159681) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_605(8159681) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_611(8159687) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_611(8159687) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_648(8159724) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_648(8159724) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_652(8159728) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_652(8159728) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_651(8159727) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_651(8159727) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_654(8159730) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_654(8159730) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_653(8159729) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_653(8159729) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_649(8159725) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_649(8159725) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_650(8159726) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_650(8159726) done Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_655(8159731) WEXITSTATUS 0 Nov 04 23:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_655(8159731) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_667(8159743) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_667(8159743) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_642(8159718) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_642(8159718) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_665(8159741) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_665(8159741) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_662(8159738) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_662(8159738) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_663(8159739) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_663(8159739) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_658(8159734) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_658(8159734) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_634(8159710) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_634(8159710) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_632(8159708) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_632(8159708) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_628(8159704) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_628(8159704) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_659(8159735) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_659(8159735) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_630(8159706) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_630(8159706) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_637(8159713) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_637(8159713) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_638(8159714) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_638(8159714) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_641(8159717) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_641(8159717) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_635(8159711) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_635(8159711) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_645(8159721) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_645(8159721) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_629(8159705) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_629(8159705) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_622(8159698) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_622(8159698) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_661(8159737) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_661(8159737) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_656(8159732) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_656(8159732) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_666(8159742) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_666(8159742) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_646(8159722) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_646(8159722) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_623(8159699) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_623(8159699) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_639(8159715) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_639(8159715) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_640(8159716) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_640(8159716) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_625(8159701) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_625(8159701) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_644(8159720) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_644(8159720) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_626(8159702) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_626(8159702) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_636(8159712) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_636(8159712) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_643(8159719) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_643(8159719) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_647(8159723) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_647(8159723) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_631(8159707) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_631(8159707) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_627(8159703) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_627(8159703) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_633(8159709) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_633(8159709) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_624(8159700) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_624(8159700) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_664(8159740) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_664(8159740) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_657(8159733) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_657(8159733) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_668(8158777) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_668(8158777) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_660(8159736) WEXITSTATUS 0 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_660(8159736) done Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5161(8158515) WEXITSTATUS 1 Nov 04 23:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5161(8158515) done Nov 04 23:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_98(8158914) WEXITSTATUS 0 Nov 04 23:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_98(8158914) done Nov 04 23:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_102(8158918) WEXITSTATUS 0 Nov 04 23:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_102(8158918) done Nov 04 23:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5069(8158381) WEXITSTATUS 1 Nov 04 23:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5069(8158381) done Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5446(8159762) in scavenge on r801u30n01 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5447(8159763) in scavenge on r801u04n03 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5448(8159764) in scavenge on r802u04n04 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5449(8159765) in scavenge on r912u05n04 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5450(8159766) in scavenge on r904u01n03 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5451(8159767) in scavenge on r916u04n01 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5452(8159768) in scavenge on r816u15n09 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5453(8159769) in scavenge on r816u15n12 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5454(8159770) in scavenge on r805u04n01 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5455(8159771) in scavenge on r909u04n03 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5456(8159772) in scavenge on r904u04n01 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5457(8159773) in scavenge on r919u32n04 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5458(8159774) in scavenge on r919u32n04 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5459(8159775) in scavenge on r913u09n03 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5460(8159776) in scavenge on r913u13n04 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5461(8159777) in scavenge on r806u02n03 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5462(8159778) in scavenge on r806u04n04 Nov 04 23:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5463(8159779) in scavenge on r806u06n01 Nov 04 23:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_145(8158984) WEXITSTATUS 0 Nov 04 23:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_145(8158984) done Nov 04 23:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_119(8158935) WEXITSTATUS 0 Nov 04 23:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_119(8158935) done Nov 04 23:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_173(8159012) WEXITSTATUS 0 Nov 04 23:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_173(8159012) done Nov 04 23:32:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_18(8159780) in scavenge on r806u11n04 Nov 04 23:32:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_19(8159781) in scavenge on r815u29n01 Nov 04 23:32:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4649(8156253) WEXITSTATUS 1 Nov 04 23:32:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4649(8156253) done Nov 04 23:32:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 04 23:32:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 04 23:32:59 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 04 23:33:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8129333 Nov 04 23:33:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_147(8158986) WEXITSTATUS 0 Nov 04 23:33:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_147(8158986) done Nov 04 23:33:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1431(8135964) in scavenge on r816u35n10 Nov 04 23:33:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_893(8128055) in scavenge on r816u35n10 Nov 04 23:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_219(8159075) WEXITSTATUS 0 Nov 04 23:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_219(8159075) done Nov 04 23:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_92(8158908) WEXITSTATUS 0 Nov 04 23:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_92(8158908) done Nov 04 23:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_68(8158871) WEXITSTATUS 0 Nov 04 23:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_68(8158871) done Nov 04 23:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_208(8159064) WEXITSTATUS 0 Nov 04 23:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_208(8159064) done Nov 04 23:33:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_566(8113946) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:33:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2297(8126245) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:33:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2625(8127883) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:33:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3408(8135235) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:33:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4842(8157850) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:33:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5018(8158309) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_149(8158988) WEXITSTATUS 0 Nov 04 23:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_149(8158988) done Nov 04 23:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5198(8158561) WEXITSTATUS 1 Nov 04 23:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5198(8158561) done Nov 04 23:33:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3408(8135235) Nov 04 23:33:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157466 WEXITSTATUS 0 Nov 04 23:33:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157466 done Nov 04 23:33:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4842(8157850) Nov 04 23:33:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5018(8158309) Nov 04 23:33:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2297(8126245) Nov 04 23:33:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2625(8127883) Nov 04 23:33:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_566(8113946) Nov 04 23:33:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_156(8158995) WEXITSTATUS 0 Nov 04 23:33:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_156(8158995) done Nov 04 23:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_138(8158977) WEXITSTATUS 0 Nov 04 23:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_138(8158977) done Nov 04 23:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_124(8158940) WEXITSTATUS 0 Nov 04 23:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_124(8158940) done Nov 04 23:33:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_159(8158998) WEXITSTATUS 0 Nov 04 23:33:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_159(8158998) done Nov 04 23:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_166(8159005) WEXITSTATUS 0 Nov 04 23:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_166(8159005) done Nov 04 23:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_143(8158982) WEXITSTATUS 0 Nov 04 23:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_143(8158982) done Nov 04 23:33:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_161(8159000) WEXITSTATUS 0 Nov 04 23:33:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_161(8159000) done Nov 04 23:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_99(8158915) WEXITSTATUS 0 Nov 04 23:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_99(8158915) done Nov 04 23:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907586 WEXITSTATUS 0 Nov 04 23:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907586 done Nov 04 23:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_211(8159067) WEXITSTATUS 0 Nov 04 23:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_211(8159067) done Nov 04 23:34:02 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_163(8159002) WEXITSTATUS 0 Nov 04 23:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_163(8159002) done Nov 04 23:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8138027 WEXITSTATUS 0 Nov 04 23:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8138027 done Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:34:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:34:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_122(8158938) WEXITSTATUS 0 Nov 04 23:34:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_122(8158938) done Nov 04 23:34:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_209(8159065) WEXITSTATUS 0 Nov 04 23:34:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_209(8159065) done Nov 04 23:34:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157467 WEXITSTATUS 0 Nov 04 23:34:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157467 done Nov 04 23:34:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_243(8159142) WEXITSTATUS 0 Nov 04 23:34:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_243(8159142) done Nov 04 23:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4760(8157331) WEXITSTATUS 1 Nov 04 23:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4760(8157331) done Nov 04 23:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_212(8159068) WEXITSTATUS 0 Nov 04 23:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_212(8159068) done Nov 04 23:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_101(8158917) WEXITSTATUS 0 Nov 04 23:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_101(8158917) done Nov 04 23:34:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_165(8159004) WEXITSTATUS 0 Nov 04 23:34:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_165(8159004) done Nov 04 23:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_155(8158994) WEXITSTATUS 0 Nov 04 23:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_155(8158994) done Nov 04 23:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_150(8158989) WEXITSTATUS 0 Nov 04 23:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_150(8158989) done Nov 04 23:34:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5127(8158467) WEXITSTATUS 1 Nov 04 23:34:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5127(8158467) done Nov 04 23:34:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5450(8159766) WEXITSTATUS 1 Nov 04 23:34:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5450(8159766) done Nov 04 23:34:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_172(8159011) WEXITSTATUS 0 Nov 04 23:34:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_172(8159011) done Nov 04 23:34:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_106(8158922) WEXITSTATUS 0 Nov 04 23:34:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_106(8158922) done Nov 04 23:34:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_128(8158944) WEXITSTATUS 0 Nov 04 23:34:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_128(8158944) done Nov 04 23:34:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_171(8159010) WEXITSTATUS 0 Nov 04 23:34:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_171(8159010) done Nov 04 23:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156071 WEXITSTATUS 0 Nov 04 23:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156071 done Nov 04 23:35:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_536(7572705) Nov 04 23:35:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_582(7572751) Nov 04 23:35:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1963(8125373) Nov 04 23:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_142(8158981) WEXITSTATUS 0 Nov 04 23:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_142(8158981) done Nov 04 23:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_157(8158996) WEXITSTATUS 0 Nov 04 23:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_157(8158996) done Nov 04 23:35:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_93(8158909) WEXITSTATUS 0 Nov 04 23:35:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_93(8158909) done Nov 04 23:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_158(8158997) WEXITSTATUS 0 Nov 04 23:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_158(8158997) done Nov 04 23:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_141(8158980) WEXITSTATUS 0 Nov 04 23:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_141(8158980) done Nov 04 23:35:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_170(8159009) WEXITSTATUS 0 Nov 04 23:35:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_170(8159009) done Nov 04 23:35:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_146(8158985) WEXITSTATUS 0 Nov 04 23:35:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_146(8158985) done Nov 04 23:35:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5438(8159562) WEXITSTATUS 1 Nov 04 23:35:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5438(8159562) done Nov 04 23:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_164(8159003) WEXITSTATUS 0 Nov 04 23:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_164(8159003) done Nov 04 23:35:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159782 InitPrio=21237 usec=8366 Nov 04 23:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_125(8158941) WEXITSTATUS 0 Nov 04 23:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_125(8158941) done Nov 04 23:35:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8159782 NodeList=r806u23n04 #CPUs=3 Partition=devel Nov 04 23:35:36 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 04 23:35:37 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 04 23:35:38 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_15(8159783) in pi_ohern on r801u32n02 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_16(8159784) in pi_ohern on r801u32n02 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_17(8159785) in pi_ohern on r801u32n02 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_18(8159786) in pi_ohern on r801u32n02 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_19(8159787) in pi_ohern on r801u32n02 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_20(8159788) in pi_ohern on r801u32n02 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_21(8159789) in pi_ohern on r801u32n02 Nov 04 23:35:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_22(8159790) in pi_ohern on r801u32n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_23(8159791) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_24(8159792) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_25(8159793) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_26(8159794) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_27(8159795) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_28(8159796) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_29(8159797) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_30(8159798) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_31(8159799) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_32(8159800) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_33(8159801) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_34(8159802) in pi_ohern on r917u13n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_35(8159803) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_36(8159804) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_37(8159805) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_38(8159806) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_39(8159807) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_40(8159808) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_41(8159809) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_42(8159810) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_43(8159811) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_44(8159812) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_45(8159813) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_46(8159814) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_47(8159815) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_48(8159816) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_49(8159817) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_50(8159818) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_51(8159819) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_52(8159820) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_53(8159821) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_54(8159822) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_55(8159823) in pi_ohern on r917u13n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_56(8159824) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_57(8159825) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_58(8159826) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_59(8159827) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_60(8159828) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_61(8159829) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_62(8159830) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_63(8159831) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_64(8159832) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_65(8159833) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_66(8159834) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_67(8159835) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_68(8159836) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_69(8159837) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_70(8159838) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_71(8159839) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_72(8159840) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_73(8159841) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_74(8159842) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_75(8159843) in pi_ohern on r917u13n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_76(8159844) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_77(8159845) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_78(8159846) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_79(8159847) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_80(8159848) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_81(8159849) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_82(8159850) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1434(8135967) WEXITSTATUS 0 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1434(8135967) done Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_83(8159851) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_84(8159852) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_85(8159853) in pi_ohern on r917u13n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_86(8159854) in pi_ohern on r917u29n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_87(8159855) in pi_ohern on r917u29n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_88(8159856) in pi_ohern on r917u29n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_89(8159857) in pi_ohern on r917u29n01 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_90(8159858) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_91(8159859) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_92(8159860) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_93(8159861) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_94(8159862) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_95(8159863) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_96(8159864) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_97(8159865) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_98(8159866) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_99(8159867) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_100(8159868) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_101(8159869) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_102(8159870) in pi_ohern on r917u29n02 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_103(8159871) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_104(8159872) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_105(8159873) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_106(8159874) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_107(8159875) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_108(8159876) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_109(8159877) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_110(8159878) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_111(8159879) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_112(8159880) in pi_ohern on r917u29n03 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_113(8159881) in pi_ohern on r917u29n04 Nov 04 23:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907587 in gpu on r805u34n01 Nov 04 23:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1855(8159882) in day on r918u32n04 Nov 04 23:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1856(8159883) in day on r918u32n04 Nov 04 23:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1857(8159884) in day on r815u09n06 Nov 04 23:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1858(8159885) in day on r815u09n06 Nov 04 23:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157710 in scavenge on r815u03n09 Nov 04 23:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157711 in scavenge on r815u09n06 Nov 04 23:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157712 in scavenge on r815u09n07 Nov 04 23:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157713 in scavenge on r815u15n03 Nov 04 23:35:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159886 InitPrio=32537 usec=5583 Nov 04 23:35:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_167(8159006) WEXITSTATUS 0 Nov 04 23:35:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_167(8159006) done Nov 04 23:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_148(8158987) WEXITSTATUS 0 Nov 04 23:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_148(8158987) done Nov 04 23:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_162(8159001) WEXITSTATUS 0 Nov 04 23:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_162(8159001) done Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5464(8159887) in scavenge on r904u01n03 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5465(8159888) in scavenge on r802u23n04 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5466(8159889) in scavenge on r802u25n01 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5467(8159890) in scavenge on r815u09n12 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5468(8159891) in scavenge on r807u24n01 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5469(8159892) in scavenge on r903u34n01 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5470(8159893) in scavenge on r913u35n01 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5471(8159894) in scavenge on r806u14n01 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5472(8159895) in scavenge on r909u29n02 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5473(8159896) in scavenge on r913u13n03 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5474(8159897) in scavenge on r918u32n04 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5475(8159898) in scavenge on r918u32n04 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5476(8159899) in scavenge on r918u32n04 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5477(8159900) in scavenge on r918u32n04 Nov 04 23:35:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5478(8159901) in scavenge on r816u35n01 Nov 04 23:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5026(8158322) WEXITSTATUS 1 Nov 04 23:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5026(8158322) done Nov 04 23:35:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_144(8158983) WEXITSTATUS 0 Nov 04 23:35:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_144(8158983) done Nov 04 23:35:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_102(7572269) in scavenge on r806u35n01 Nov 04 23:36:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_154(8158993) WEXITSTATUS 0 Nov 04 23:36:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_154(8158993) done Nov 04 23:36:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159902 InitPrio=32537 usec=4643 Nov 04 23:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157471 WEXITSTATUS 0 Nov 04 23:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157471 done Nov 04 23:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_210(8159066) WEXITSTATUS 0 Nov 04 23:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_210(8159066) done Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159903 InitPrio=32537 usec=2412 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159904 InitPrio=32537 usec=4041 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159905 InitPrio=32537 usec=3022 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159906 InitPrio=32537 usec=3403 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159907 InitPrio=32537 usec=2692 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159908 InitPrio=32537 usec=2530 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159909 InitPrio=32537 usec=2778 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159910 InitPrio=32537 usec=2483 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159911 InitPrio=32537 usec=2587 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159912 InitPrio=32537 usec=2598 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159913 InitPrio=32537 usec=2516 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159914 InitPrio=32537 usec=2552 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159915 InitPrio=32537 usec=2497 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159916 InitPrio=32537 usec=2797 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159917 InitPrio=32537 usec=2480 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159918 InitPrio=32537 usec=2711 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159919 InitPrio=32537 usec=2406 Nov 04 23:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159920 InitPrio=32537 usec=2160 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5275(8158709) WEXITSTATUS 1 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5275(8158709) done Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159921 InitPrio=32537 usec=4264 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159922 InitPrio=32537 usec=4055 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159923 InitPrio=32537 usec=3223 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159924 InitPrio=32537 usec=3847 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159925 InitPrio=32537 usec=3096 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159926 InitPrio=32537 usec=3175 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159927 InitPrio=32537 usec=2478 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159928 InitPrio=32537 usec=2836 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159929 InitPrio=32537 usec=2311 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159930 InitPrio=32537 usec=2643 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159931 InitPrio=32537 usec=2430 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159932 InitPrio=32537 usec=2525 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159933 InitPrio=32537 usec=2669 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159934 InitPrio=32537 usec=2377 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159935 InitPrio=32537 usec=2591 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159936 InitPrio=32537 usec=2290 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159937 InitPrio=32537 usec=2556 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159938 InitPrio=32537 usec=2448 Nov 04 23:36:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159939 InitPrio=32537 usec=2250 Nov 04 23:36:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159940 InitPrio=32537 usec=4361 Nov 04 23:36:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_179(8159035) WEXITSTATUS 0 Nov 04 23:36:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_179(8159035) done Nov 04 23:36:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4711(8156722) WEXITSTATUS 1 Nov 04 23:36:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4711(8156722) done Nov 04 23:36:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4892(8158003) WEXITSTATUS 1 Nov 04 23:36:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4892(8158003) done Nov 04 23:36:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1458(8137918) in scavenge on r815u03n12 Nov 04 23:36:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1461(8138138) in scavenge on r815u03n12 Nov 04 23:36:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1460(8138137) in scavenge on r913u35n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157468 WEXITSTATUS 0 Nov 04 23:36:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157468 done Nov 04 23:36:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_218(8159074) WEXITSTATUS 0 Nov 04 23:36:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_218(8159074) done Nov 04 23:36:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4982(8158244) WEXITSTATUS 1 Nov 04 23:36:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4982(8158244) done Nov 04 23:36:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_220(8159076) WEXITSTATUS 0 Nov 04 23:36:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_220(8159076) done Nov 04 23:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5124(8158464) WEXITSTATUS 1 Nov 04 23:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5124(8158464) done Nov 04 23:36:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_66(8158869) WEXITSTATUS 0 Nov 04 23:36:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_66(8158869) done Nov 04 23:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_280(8159246) WEXITSTATUS 0 Nov 04 23:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_280(8159246) done Nov 04 23:36:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_16(8154078) Nov 04 23:36:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_17(8154079) Nov 04 23:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5083(8158407) WEXITSTATUS 1 Nov 04 23:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5083(8158407) done Nov 04 23:36:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_8(8158859) WEXITSTATUS 0 Nov 04 23:36:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_8(8158859) done Nov 04 23:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4152(8150371) WEXITSTATUS 1 Nov 04 23:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4152(8150371) done Nov 04 23:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_10(8158776) WEXITSTATUS 0 Nov 04 23:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_10(8158776) done Nov 04 23:36:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_180(8159036) WEXITSTATUS 0 Nov 04 23:36:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_180(8159036) done Nov 04 23:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3602(8159759) WEXITSTATUS 0 Nov 04 23:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3602(8159759) done Nov 04 23:36:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3596(8159547) WEXITSTATUS 0 Nov 04 23:36:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3596(8159547) done Nov 04 23:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157473 WEXITSTATUS 0 Nov 04 23:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157473 done Nov 04 23:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_7(8158858) WEXITSTATUS 0 Nov 04 23:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_7(8158858) done Nov 04 23:37:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_247(8159146) WEXITSTATUS 0 Nov 04 23:37:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_247(8159146) done Nov 04 23:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_9(8158860) WEXITSTATUS 0 Nov 04 23:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_9(8158860) done Nov 04 23:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_6(8158857) WEXITSTATUS 0 Nov 04 23:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_6(8158857) done Nov 04 23:37:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157714 NodeList=r815u23n09 #CPUs=12 Partition=scavenge Nov 04 23:37:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157715 NodeList=r815u23n12 #CPUs=12 Partition=scavenge Nov 04 23:37:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157716 NodeList=r815u23n12 #CPUs=12 Partition=scavenge Nov 04 23:37:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157717 NodeList=r815u23n12 #CPUs=12 Partition=scavenge Nov 04 23:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_281(8159247) WEXITSTATUS 0 Nov 04 23:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_281(8159247) done Nov 04 23:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_285(8159251) WEXITSTATUS 0 Nov 04 23:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_285(8159251) done Nov 04 23:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_284(8159250) WEXITSTATUS 0 Nov 04 23:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_284(8159250) done Nov 04 23:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_178(8159034) WEXITSTATUS 0 Nov 04 23:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_178(8159034) done Nov 04 23:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_177(8159033) WEXITSTATUS 0 Nov 04 23:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_177(8159033) done Nov 04 23:37:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_205(8159061) WEXITSTATUS 0 Nov 04 23:37:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_205(8159061) done Nov 04 23:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3522(8158227) WEXITSTATUS 0 Nov 04 23:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3522(8158227) done Nov 04 23:37:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_203(8159059) WEXITSTATUS 0 Nov 04 23:37:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_203(8159059) done Nov 04 23:37:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157472 WEXITSTATUS 0 Nov 04 23:37:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157472 done Nov 04 23:37:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_183(8159039) WEXITSTATUS 0 Nov 04 23:37:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_183(8159039) done Nov 04 23:37:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_254(8159153) WEXITSTATUS 0 Nov 04 23:37:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_254(8159153) done Nov 04 23:37:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8159941 InitPrio=23571 usec=4109 Nov 04 23:37:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_214(8159070) WEXITSTATUS 0 Nov 04 23:37:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_214(8159070) done Nov 04 23:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_239(8159138) WEXITSTATUS 0 Nov 04 23:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_239(8159138) done Nov 04 23:38:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_176(8159032) WEXITSTATUS 0 Nov 04 23:38:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_176(8159032) done Nov 04 23:38:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4962(8158212) WEXITSTATUS 1 Nov 04 23:38:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4962(8158212) done Nov 04 23:38:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_277(8159243) WEXITSTATUS 0 Nov 04 23:38:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_277(8159243) done Nov 04 23:38:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_5(8158856) WEXITSTATUS 0 Nov 04 23:38:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_5(8158856) done Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:38:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:38:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_200(8159056) WEXITSTATUS 0 Nov 04 23:38:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_200(8159056) done Nov 04 23:38:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4986(8158259) WEXITSTATUS 1 Nov 04 23:38:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4986(8158259) done Nov 04 23:38:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156072 WEXITSTATUS 0 Nov 04 23:38:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156072 done Nov 04 23:38:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_241(8159140) WEXITSTATUS 0 Nov 04 23:38:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_241(8159140) done Nov 04 23:38:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157718 NodeList=r815u09n12 #CPUs=12 Partition=scavenge Nov 04 23:38:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157719 NodeList=r815u15n10 #CPUs=12 Partition=scavenge Nov 04 23:38:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_215(8159071) WEXITSTATUS 0 Nov 04 23:38:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_215(8159071) done Nov 04 23:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_175(8159031) WEXITSTATUS 0 Nov 04 23:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_175(8159031) done Nov 04 23:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5442(8159657) WEXITSTATUS 1 Nov 04 23:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5442(8159657) done Nov 04 23:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4215(8152270) WEXITSTATUS 1 Nov 04 23:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4215(8152270) done Nov 04 23:38:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_224(8159123) WEXITSTATUS 0 Nov 04 23:38:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_224(8159123) done Nov 04 23:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_204(8159060) WEXITSTATUS 0 Nov 04 23:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_204(8159060) done Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_114(8159942) in pi_ohern on r801u32n02 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_115(8159943) in pi_ohern on r801u32n02 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_116(8159944) in pi_ohern on r801u32n02 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_117(8159945) in pi_ohern on r801u32n02 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_118(8159946) in pi_ohern on r801u32n02 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_119(8159947) in pi_ohern on r801u32n02 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_120(8159948) in pi_ohern on r801u32n02 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_121(8159949) in pi_ohern on r917u13n04 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_122(8159950) in pi_ohern on r917u29n01 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_123(8159951) in pi_ohern on r917u29n04 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_124(8159952) in pi_ohern on r917u29n04 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_125(8159953) in pi_ohern on r917u29n04 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_126(8159954) in pi_ohern on r917u29n04 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_127(8159955) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_128(8159956) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_129(8159957) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_130(8159958) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_131(8159959) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_132(8159960) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_133(8159961) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_134(8159962) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_135(8159963) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_136(8159964) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_137(8159965) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_138(8159966) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_139(8159967) in pi_ohern on r815u03n11 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_140(8159968) in pi_ohern on r815u03n12 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_141(8159969) in pi_ohern on r815u03n12 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_142(8159970) in pi_ohern on r911u32n01 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_143(8159971) in pi_ohern on r911u32n01 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_144(8159972) in pi_ohern on r911u32n01 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_145(8159973) in pi_ohern on r911u32n01 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_146(8159974) in pi_ohern on r911u32n01 Nov 04 23:38:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_147(8159975) in pi_ohern on r911u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_148(8159976) in pi_ohern on r911u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_149(8159977) in pi_ohern on r911u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_150(8159978) in pi_ohern on r911u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_151(8159979) in pi_ohern on r911u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_152(8159980) in pi_ohern on r911u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_153(8159981) in pi_ohern on r911u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_154(8159982) in pi_ohern on r807u22n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_155(8159983) in pi_ohern on r807u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_156(8159984) in pi_ohern on r807u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_157(8159985) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_158(8159986) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_159(8159987) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_160(8159988) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_161(8159989) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_162(8159990) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_163(8159991) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_164(8159992) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_165(8159993) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_166(8159994) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_167(8159995) in pi_ohern on r903u12n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_168(8159996) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_169(8159997) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_170(8159998) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_171(8159999) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_172(8160000) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_173(8160001) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_174(8160002) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_175(8160003) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_176(8160004) in pi_ohern on r903u24n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_177(8160005) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_178(8160006) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_179(8160007) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_180(8160008) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_181(8160009) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_182(8160010) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_183(8160011) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_184(8160012) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_185(8160013) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_186(8160014) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_187(8160015) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_188(8160016) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_189(8160017) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_190(8160018) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_191(8160019) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_192(8160020) in pi_ohern on r913u32n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_193(8160021) in pi_ohern on r913u35n01 Nov 04 23:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_194(8160022) in pi_ohern on r913u35n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_0(8160023) in day on r906u41n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_1(8160024) in day on r906u41n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_2(8160025) in day on r906u41n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_3(8160026) in day on r906u41n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_4(8160027) in day on r906u41n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_5(8160028) in day on r906u41n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_6(8160029) in day on r906u41n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_7(8160030) in day on r906u41n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_8(8160031) in day on r906u41n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_9(8160032) in day on r904u04n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_10(8160033) in day on r904u04n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_11(8160034) in day on r904u04n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_12(8160035) in day on r904u04n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_13(8160036) in day on r904u04n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_14(8160037) in day on r904u04n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_15(8160038) in day on r904u04n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_16(8160039) in day on r904u04n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_17(8160040) in day on r904u04n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_18(8160041) in day on r904u04n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_19(8160042) in day on r904u04n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_20(8160043) in day on r904u04n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_21(8160044) in day on r904u04n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_22(8160045) in day on r904u04n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_23(8160046) in day on r904u04n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_24(8160047) in day on r904u04n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_25(8160048) in day on r904u04n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_26(8160049) in day on r904u07n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_27(8160050) in day on r904u07n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_28(8160051) in day on r904u07n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_29(8160052) in day on r904u07n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_30(8160053) in day on r904u07n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_31(8160054) in day on r904u07n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_32(8160055) in day on r904u07n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_33(8160056) in day on r904u07n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_34(8160057) in day on r904u07n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_35(8160058) in day on r904u07n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_36(8160059) in day on r904u07n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_37(8160060) in day on r904u07n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_38(8160061) in day on r904u07n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_39(8160062) in day on r904u07n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_40(8160063) in day on r904u07n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_41(8160064) in day on r904u07n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_42(8160065) in day on r904u07n04 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_43(8160066) in day on r906u41n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_44(8160067) in day on r906u41n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_45(8160068) in day on r906u41n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_46(8160069) in day on r906u41n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_47(8160070) in day on r904u10n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_48(8160071) in day on r904u10n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_49(8160072) in day on r904u10n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_50(8160073) in day on r904u10n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_51(8160074) in day on r904u10n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_52(8160075) in day on r904u10n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_53(8160076) in day on r904u10n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_54(8160077) in day on r904u10n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_55(8160078) in day on r914u01n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_56(8160079) in day on r914u01n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_57(8160080) in day on r914u05n01 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_58(8160081) in day on r914u05n02 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_59(8160082) in day on r914u05n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_60(8160083) in day on r914u05n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_61(8160084) in day on r914u05n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_62(8160085) in day on r914u05n03 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_791(7572960) WEXITSTATUS 0 Nov 04 23:38:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_791(7572960) done Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_63(8160086) in day on r914u05n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_64(8160087) in day on r914u05n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_65(8160088) in day on r914u05n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_66(8160089) in day on r914u05n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_67(8160090) in day on r914u05n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_68(8160091) in day on r914u05n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_69(8160092) in day on r914u05n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_70(8160093) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_71(8160094) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_72(8160095) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_73(8160096) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_74(8160097) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_75(8160098) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_76(8160099) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_77(8160100) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_78(8160101) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_79(8160102) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_80(8160103) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_81(8160104) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_82(8160105) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_83(8160106) in day on r807u03n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_84(8160107) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_85(8160108) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_86(8160109) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_87(8160110) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_88(8160111) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_89(8160112) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_90(8160113) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_91(8160114) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_92(8160115) in day on r807u03n03 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_93(8160116) in day on r807u03n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_94(8160117) in day on r807u03n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_95(8160118) in day on r807u03n04 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_96(8160119) in day on r807u05n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_97(8160120) in day on r807u05n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_98(8160121) in day on r807u05n02 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_99(8160122) in day on r807u05n02 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3603(8160123) in gpu on r808u14n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3604(8160124) in gpu on r4513u20n01 Nov 04 23:38:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3605(8160125) in gpu on r4513u23n01 Nov 04 23:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_188(8159044) WEXITSTATUS 0 Nov 04 23:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_188(8159044) done Nov 04 23:38:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1859(8160126) in day on r816u03n02 Nov 04 23:38:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1860(8160127) in day on r816u03n02 Nov 04 23:38:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r914u01n02 Nov 04 23:38:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157720 in scavenge on r815u15n05 Nov 04 23:38:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157721 in scavenge on r816u03n06 Nov 04 23:38:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157722 in scavenge on r816u09n02 Nov 04 23:38:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157723 in scavenge on r816u09n07 Nov 04 23:38:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157724 in scavenge on r816u15n01 Nov 04 23:38:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_174(8159030) WEXITSTATUS 0 Nov 04 23:38:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_174(8159030) done Nov 04 23:38:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1253(8135155) WEXITSTATUS 0 Nov 04 23:38:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1253(8135155) done Nov 04 23:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_185(8159041) WEXITSTATUS 0 Nov 04 23:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_185(8159041) done Nov 04 23:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1415(8135948) WEXITSTATUS 0 Nov 04 23:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1415(8135948) done Nov 04 23:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157480 WEXITSTATUS 0 Nov 04 23:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157480 done Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5479(8160128) in scavenge on r907u05n03 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5480(8160129) in scavenge on r801u09n04 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5481(8160130) in scavenge on r903u04n01 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5482(8160131) in scavenge on r802u06n02 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5483(8160132) in scavenge on r802u23n02 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5484(8160133) in scavenge on r917u10n01 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5485(8160134) in scavenge on r917u10n04 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5486(8160135) in scavenge on r815u29n11 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5487(8160136) in scavenge on r815u09n10 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5488(8160137) in scavenge on r815u15n12 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5489(8160138) in scavenge on r806u14n03 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5490(8160139) in scavenge on r909u10n01 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5491(8160140) in scavenge on r919u05n03 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3599(8159652) WEXITSTATUS 0 Nov 04 23:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3599(8159652) done Nov 04 23:39:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_202(8159058) WEXITSTATUS 0 Nov 04 23:39:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_202(8159058) done Nov 04 23:39:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 04 23:39:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 04 23:39:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 04 23:39:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_114(7572281) in scavenge on r815u23n10 Nov 04 23:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_160(8158999) WEXITSTATUS 0 Nov 04 23:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_160(8158999) done Nov 04 23:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_276(8159242) WEXITSTATUS 0 Nov 04 23:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_276(8159242) done Nov 04 23:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3603(8160123) WEXITSTATUS 0 Nov 04 23:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3603(8160123) done Nov 04 23:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157477 WEXITSTATUS 0 Nov 04 23:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157477 done Nov 04 23:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_216(8159072) WEXITSTATUS 0 Nov 04 23:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_216(8159072) done Nov 04 23:39:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160141 InitPrio=40248 usec=6647 Nov 04 23:39:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157470 WEXITSTATUS 0 Nov 04 23:39:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157470 done Nov 04 23:39:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_198(8159054) WEXITSTATUS 0 Nov 04 23:39:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_198(8159054) done Nov 04 23:39:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_279(8159245) WEXITSTATUS 0 Nov 04 23:39:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_279(8159245) done Nov 04 23:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_196(8159052) WEXITSTATUS 0 Nov 04 23:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_196(8159052) done Nov 04 23:39:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_238(8159137) WEXITSTATUS 0 Nov 04 23:39:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_238(8159137) done Nov 04 23:39:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3605(8160125) WEXITSTATUS 0 Nov 04 23:39:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3605(8160125) done Nov 04 23:39:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_181(8159037) WEXITSTATUS 0 Nov 04 23:39:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_181(8159037) done Nov 04 23:39:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5248(8158672) WEXITSTATUS 1 Nov 04 23:39:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5248(8158672) done Nov 04 23:39:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_226(8159125) WEXITSTATUS 0 Nov 04 23:39:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_226(8159125) done Nov 04 23:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_257(8159156) WEXITSTATUS 0 Nov 04 23:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_257(8159156) done Nov 04 23:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_201(8159057) WEXITSTATUS 0 Nov 04 23:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_201(8159057) done Nov 04 23:39:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157469 WEXITSTATUS 0 Nov 04 23:39:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157469 done Nov 04 23:39:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160141 NodeList=r914u01n02 #CPUs=8 Partition=day Nov 04 23:39:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157725 NodeList=r815u23n08 #CPUs=12 Partition=scavenge Nov 04 23:39:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157726 NodeList=r816u23n08 #CPUs=12 Partition=scavenge Nov 04 23:39:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_229(8159128) WEXITSTATUS 0 Nov 04 23:39:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_229(8159128) done Nov 04 23:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157478 WEXITSTATUS 0 Nov 04 23:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157478 done Nov 04 23:39:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2413(8126655) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:39:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3444(8135292) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:39:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3461(8135329) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:39:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1458(8137918) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:39:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1460(8138137) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:39:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_184(8159040) WEXITSTATUS 0 Nov 04 23:39:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_184(8159040) done Nov 04 23:39:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3444(8135292) Nov 04 23:39:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1460(8138137) Nov 04 23:39:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3461(8135329) Nov 04 23:39:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2413(8126655) Nov 04 23:39:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1458(8137918) Nov 04 23:39:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5062(8158367) WEXITSTATUS 1 Nov 04 23:39:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5062(8158367) done Nov 04 23:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5169(8158523) WEXITSTATUS 1 Nov 04 23:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5169(8158523) done Nov 04 23:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_213(8159069) WEXITSTATUS 0 Nov 04 23:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_213(8159069) done Nov 04 23:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5307(8158769) WEXITSTATUS 1 Nov 04 23:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5307(8158769) done Nov 04 23:39:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_259(8159158) WEXITSTATUS 0 Nov 04 23:39:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_259(8159158) done Nov 04 23:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_187(8159043) WEXITSTATUS 0 Nov 04 23:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_187(8159043) done Nov 04 23:40:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_233(8159132) WEXITSTATUS 0 Nov 04 23:40:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_233(8159132) done Nov 04 23:40:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160142 InitPrio=17454 usec=3852 Nov 04 23:40:04 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160142 NodeList=r801u04n01 #CPUs=4 Partition=pi_esi Nov 04 23:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_3(8158854) WEXITSTATUS 0 Nov 04 23:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_3(8158854) done Nov 04 23:40:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_283(8159249) WEXITSTATUS 0 Nov 04 23:40:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_283(8159249) done Nov 04 23:40:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157474 WEXITSTATUS 0 Nov 04 23:40:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157474 done Nov 04 23:40:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_199(8159055) WEXITSTATUS 0 Nov 04 23:40:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_199(8159055) done Nov 04 23:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_242(8159141) WEXITSTATUS 0 Nov 04 23:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_242(8159141) done Nov 04 23:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_168(8159007) WEXITSTATUS 0 Nov 04 23:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_168(8159007) done Nov 04 23:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_245(8159144) WEXITSTATUS 0 Nov 04 23:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_245(8159144) done Nov 04 23:40:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_2(8158853) WEXITSTATUS 0 Nov 04 23:40:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_2(8158853) done Nov 04 23:40:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157481 WEXITSTATUS 0 Nov 04 23:40:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157481 done Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_186(8159042) WEXITSTATUS 0 Nov 04 23:40:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_186(8159042) done Nov 04 23:40:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_228(8159127) WEXITSTATUS 0 Nov 04 23:40:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_228(8159127) done Nov 04 23:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_193(8159049) WEXITSTATUS 0 Nov 04 23:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_193(8159049) done Nov 04 23:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_195(8159051) WEXITSTATUS 0 Nov 04 23:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_195(8159051) done Nov 04 23:40:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157727 NodeList=r815u15n12 #CPUs=12 Partition=scavenge Nov 04 23:40:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157728 NodeList=r816u15n10 #CPUs=12 Partition=scavenge Nov 04 23:40:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157729 NodeList=r816u23n11 #CPUs=12 Partition=scavenge Nov 04 23:40:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_4(8158855) WEXITSTATUS 0 Nov 04 23:40:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158776_4(8158855) done Nov 04 23:40:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_217(8159073) WEXITSTATUS 0 Nov 04 23:40:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_217(8159073) done Nov 04 23:40:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1167(7572134) Nov 04 23:40:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_18(8154206) Nov 04 23:40:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_19(8154207) Nov 04 23:40:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 23:40:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_282(8159248) WEXITSTATUS 0 Nov 04 23:40:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_282(8159248) done Nov 04 23:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_227(8159126) WEXITSTATUS 0 Nov 04 23:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_227(8159126) done Nov 04 23:40:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_252(8159151) WEXITSTATUS 0 Nov 04 23:40:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_252(8159151) done Nov 04 23:40:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5284(8158732) WEXITSTATUS 1 Nov 04 23:40:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5284(8158732) done Nov 04 23:40:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_191(8159047) WEXITSTATUS 0 Nov 04 23:40:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_191(8159047) done Nov 04 23:40:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_574(8159635) WEXITSTATUS 0 Nov 04 23:40:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_574(8159635) done Nov 04 23:40:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_287(8159253) WEXITSTATUS 0 Nov 04 23:40:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_287(8159253) done Nov 04 23:40:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_246(8159145) WEXITSTATUS 0 Nov 04 23:40:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_246(8159145) done Nov 04 23:40:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_182(8159038) WEXITSTATUS 0 Nov 04 23:40:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_182(8159038) done Nov 04 23:40:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4806(8157752) WEXITSTATUS 1 Nov 04 23:40:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4806(8157752) done Nov 04 23:40:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5451(8159767) WEXITSTATUS 1 Nov 04 23:40:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5451(8159767) done Nov 04 23:41:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_140(8158979) WEXITSTATUS 0 Nov 04 23:41:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_140(8158979) done Nov 04 23:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5156(8158503) WEXITSTATUS 1 Nov 04 23:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5156(8158503) done Nov 04 23:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_234(8159133) WEXITSTATUS 0 Nov 04 23:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_234(8159133) done Nov 04 23:41:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_197(8159053) WEXITSTATUS 0 Nov 04 23:41:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_197(8159053) done Nov 04 23:41:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_289(8159255) WEXITSTATUS 0 Nov 04 23:41:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_289(8159255) done Nov 04 23:41:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2182(8151974) WEXITSTATUS 0 Nov 04 23:41:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2182(8151974) done Nov 04 23:41:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_206(8159062) WEXITSTATUS 0 Nov 04 23:41:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_206(8159062) done Nov 04 23:41:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156075 WEXITSTATUS 0 Nov 04 23:41:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156075 done Nov 04 23:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_225(8159124) WEXITSTATUS 0 Nov 04 23:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_225(8159124) done Nov 04 23:41:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5181(8158535) WEXITSTATUS 1 Nov 04 23:41:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5181(8158535) done Nov 04 23:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_251(8159150) WEXITSTATUS 0 Nov 04 23:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_251(8159150) done Nov 04 23:41:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157730 NodeList=r816u15n10 #CPUs=12 Partition=scavenge Nov 04 23:41:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4705(8156588) WEXITSTATUS 1 Nov 04 23:41:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4705(8156588) done Nov 04 23:41:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5115(8158448) WEXITSTATUS 1 Nov 04 23:41:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5115(8158448) done Nov 04 23:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_221(8159120) WEXITSTATUS 0 Nov 04 23:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_221(8159120) done Nov 04 23:41:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_322(8159306) WEXITSTATUS 0 Nov 04 23:41:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_322(8159306) done Nov 04 23:41:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157479 WEXITSTATUS 0 Nov 04 23:41:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157479 done Nov 04 23:41:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_278(8159244) WEXITSTATUS 0 Nov 04 23:41:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_278(8159244) done Nov 04 23:41:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_222(8159121) WEXITSTATUS 0 Nov 04 23:41:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_222(8159121) done Nov 04 23:41:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_256(8159155) WEXITSTATUS 0 Nov 04 23:41:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_256(8159155) done Nov 04 23:41:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_288(8159254) WEXITSTATUS 0 Nov 04 23:41:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_288(8159254) done Nov 04 23:41:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_286(8159252) WEXITSTATUS 0 Nov 04 23:41:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_286(8159252) done Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_195(8160143) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_196(8160144) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_197(8160145) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_198(8160146) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_199(8160147) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_200(8160148) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_201(8160149) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_202(8160150) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_203(8160151) in pi_ohern on r801u32n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_204(8160152) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_205(8160153) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_206(8160154) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_207(8160155) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_208(8160156) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_209(8160157) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_210(8160158) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_211(8160159) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_212(8160160) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_213(8160161) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_214(8160162) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_215(8160163) in pi_ohern on r917u13n02 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_216(8160164) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_217(8160165) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_218(8160166) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_219(8160167) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_220(8160168) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_221(8160169) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_222(8160170) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_223(8160171) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_224(8160172) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_225(8160173) in pi_ohern on r917u13n03 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_226(8160174) in pi_ohern on r917u13n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_227(8160175) in pi_ohern on r917u13n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_228(8160176) in pi_ohern on r917u13n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_229(8160177) in pi_ohern on r917u13n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_230(8160178) in pi_ohern on r917u13n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_231(8160179) in pi_ohern on r917u29n01 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_232(8160180) in pi_ohern on r917u29n01 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_233(8160181) in pi_ohern on r917u29n01 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_234(8160182) in pi_ohern on r917u29n01 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_235(8160183) in pi_ohern on r917u29n01 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_236(8160184) in pi_ohern on r917u29n01 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_237(8160185) in pi_ohern on r917u29n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_238(8160186) in pi_ohern on r917u29n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_239(8160187) in pi_ohern on r917u29n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_240(8160188) in pi_ohern on r917u29n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_241(8160189) in pi_ohern on r917u29n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_242(8160190) in pi_ohern on r917u29n04 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_243(8160191) in pi_ohern on r815u03n11 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_244(8160192) in pi_ohern on r815u03n11 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_245(8160193) in pi_ohern on r815u03n11 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_246(8160194) in pi_ohern on r815u03n11 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_247(8160195) in pi_ohern on r815u03n11 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_248(8160196) in pi_ohern on r815u03n12 Nov 04 23:41:53 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:108 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_249(8160197) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_250(8160198) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_251(8160199) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_252(8160200) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_253(8160201) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_254(8160202) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_255(8160203) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_256(8160204) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_257(8160205) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_258(8160206) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_259(8160207) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_260(8160208) in pi_ohern on r815u03n12 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_261(8160209) in pi_ohern on r807u22n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_262(8160210) in pi_ohern on r807u22n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_263(8160211) in pi_ohern on r911u32n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_264(8160212) in pi_ohern on r911u32n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_265(8160213) in pi_ohern on r911u32n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_266(8160214) in pi_ohern on r911u32n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_267(8160215) in pi_ohern on r913u35n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_268(8160216) in pi_ohern on r913u35n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_269(8160217) in pi_ohern on r913u35n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_270(8160218) in pi_ohern on r903u12n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_271(8160219) in pi_ohern on r903u24n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_272(8160220) in pi_ohern on r903u24n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_273(8160221) in pi_ohern on r903u24n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_274(8160222) in pi_ohern on r903u24n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_275(8160223) in pi_ohern on r903u24n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_276(8160224) in pi_ohern on r903u24n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_277(8160225) in pi_ohern on r903u26n01 Nov 04 23:41:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159903 in day on r815u29n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_100(8160226) in day on r906u41n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_101(8160227) in day on r807u05n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_102(8160228) in day on r807u05n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_103(8160229) in day on r807u05n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_104(8160230) in day on r807u05n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_105(8160231) in day on r807u05n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_106(8160232) in day on r807u05n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_107(8160233) in day on r807u05n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_108(8160234) in day on r807u05n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_109(8160235) in day on r807u05n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_110(8160236) in day on r807u05n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_111(8160237) in day on r807u05n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_112(8160238) in day on r807u05n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_113(8160239) in day on r807u05n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_114(8160240) in day on r807u08n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_115(8160241) in day on r807u08n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_116(8160242) in day on r807u08n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_117(8160243) in day on r807u08n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_118(8160244) in day on r807u08n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_119(8160245) in day on r807u08n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_120(8160246) in day on r807u08n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_121(8160247) in day on r807u08n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_122(8160248) in day on r807u08n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_123(8160249) in day on r807u08n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_124(8160250) in day on r807u08n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_125(8160251) in day on r807u08n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_126(8160252) in day on r807u08n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_127(8160253) in day on r807u08n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_128(8160254) in day on r807u08n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_129(8160255) in day on r807u08n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_130(8160256) in day on r807u08n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_131(8160257) in day on r807u08n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_132(8160258) in day on r807u08n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_133(8160259) in day on r807u11n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_134(8160260) in day on r807u11n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_135(8160261) in day on r807u11n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_136(8160262) in day on r807u11n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_137(8160263) in day on r807u11n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_138(8160264) in day on r807u11n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_139(8160265) in day on r807u11n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_140(8160266) in day on r807u11n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_141(8160267) in day on r807u11n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_142(8160268) in day on r807u11n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_143(8160269) in day on r807u11n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_144(8160270) in day on r807u11n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_145(8160271) in day on r807u11n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_146(8160272) in day on r807u11n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_147(8160273) in day on r806u02n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_148(8160274) in day on r806u02n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_149(8160275) in day on r806u02n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_150(8160276) in day on r806u02n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_151(8160277) in day on r806u02n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_152(8160278) in day on r806u04n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_153(8160279) in day on r806u04n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_154(8160280) in day on r806u04n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_155(8160281) in day on r806u04n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_156(8160282) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_157(8160283) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_158(8160284) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_159(8160285) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_160(8160286) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_161(8160287) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_162(8160288) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_163(8160289) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_164(8160290) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_165(8160291) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_166(8160292) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_167(8160293) in day on r806u04n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_168(8160294) in day on r806u04n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_169(8160295) in day on r806u04n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_170(8160296) in day on r806u04n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_171(8160297) in day on r806u04n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_172(8160298) in day on r806u04n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_173(8160299) in day on r806u06n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_174(8160300) in day on r806u06n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_175(8160301) in day on r806u06n01 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_176(8160302) in day on r806u06n02 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_177(8160303) in day on r806u06n03 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_178(8160304) in day on r806u06n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_179(8160305) in day on r806u06n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_180(8160306) in day on r806u06n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_181(8160307) in day on r806u06n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_182(8160308) in day on r806u06n04 Nov 04 23:41:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_183(8160309) in day on r806u09n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_184(8160310) in day on r806u09n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_185(8160311) in day on r806u09n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_186(8160312) in day on r806u09n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_187(8160313) in day on r806u09n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_188(8160314) in day on r806u09n02 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_189(8160315) in day on r806u09n02 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_190(8160316) in day on r806u09n02 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_191(8160317) in day on r806u09n02 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_192(8160318) in day on r806u09n02 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_193(8160319) in day on r806u09n03 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_194(8160320) in day on r806u09n03 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_195(8160321) in day on r806u09n03 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_196(8160322) in day on r806u09n03 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_197(8160323) in day on r806u09n03 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_198(8160324) in day on r806u11n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_199(8160325) in day on r806u11n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3606(8160326) in gpu on r808u14n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3607(8160327) in gpu on r808u17n01 Nov 04 23:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3608(8160328) in gpu on r4513u23n01 Nov 04 23:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_189(8159045) WEXITSTATUS 0 Nov 04 23:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_189(8159045) done Nov 04 23:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157731 in scavenge on r815u35n02 Nov 04 23:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8157732 in scavenge on r815u35n02 Nov 04 23:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5203(8158566) WEXITSTATUS 1 Nov 04 23:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5203(8158566) done Nov 04 23:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156073 WEXITSTATUS 0 Nov 04 23:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156073 done Nov 04 23:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157486 WEXITSTATUS 0 Nov 04 23:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157486 done Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5492(8160329) in scavenge on r801u14n03 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5493(8160330) in scavenge on r916u04n01 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5494(8160331) in scavenge on r815u23n09 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5495(8160332) in scavenge on r808u08n01 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5496(8160333) in scavenge on r903u40n01 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5497(8160334) in scavenge on r805u09n01 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5498(8160335) in scavenge on r805u11n01 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5499(8160336) in scavenge on r909u07n02 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5500(8160337) in scavenge on r909u29n04 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5501(8160338) in scavenge on r919u05n04 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5502(8160339) in scavenge on r919u32n03 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5503(8160340) in scavenge on r816u03n01 Nov 04 23:42:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5504(8160341) in scavenge on r816u03n01 Nov 04 23:42:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_248(8159147) WEXITSTATUS 0 Nov 04 23:42:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_248(8159147) done Nov 04 23:42:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_20(8160342) in scavenge on r904u01n01 Nov 04 23:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_250(8159149) WEXITSTATUS 0 Nov 04 23:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_250(8159149) done Nov 04 23:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_190(8159046) WEXITSTATUS 0 Nov 04 23:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_190(8159046) done Nov 04 23:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_194(8159050) WEXITSTATUS 0 Nov 04 23:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_194(8159050) done Nov 04 23:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5466(8159889) WEXITSTATUS 1 Nov 04 23:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5466(8159889) done Nov 04 23:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_318(8159302) WEXITSTATUS 0 Nov 04 23:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_318(8159302) done Nov 04 23:42:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160343 InitPrio=17454 usec=2738 Nov 04 23:42:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_244(8159143) WEXITSTATUS 0 Nov 04 23:42:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_244(8159143) done Nov 04 23:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160343 NodeList=r801u11n02 #CPUs=4 Partition=pi_esi Nov 04 23:42:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160344 InitPrio=17454 usec=4196 Nov 04 23:42:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_223(8159122) WEXITSTATUS 0 Nov 04 23:42:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_223(8159122) done Nov 04 23:42:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160344 NodeList=r801u23n02 #CPUs=4 Partition=pi_esi Nov 04 23:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5461(8159777) WEXITSTATUS 1 Nov 04 23:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5461(8159777) done Nov 04 23:42:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5114(8158447) WEXITSTATUS 1 Nov 04 23:42:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5114(8158447) done Nov 04 23:42:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_292(8159258) WEXITSTATUS 0 Nov 04 23:42:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_292(8159258) done Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:42:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 23:42:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3594(8136426) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:42:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1461(8138138) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:42:33 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8158658_19(8159781) had to be killed Nov 04 23:42:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5468(8159891) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:42:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5470(8159893) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:42:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3594(8136426) Nov 04 23:42:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5468(8159891) Nov 04 23:42:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1461(8138138) Nov 04 23:42:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5470(8159893) Nov 04 23:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5445(8159660) WEXITSTATUS 1 Nov 04 23:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5445(8159660) done Nov 04 23:42:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_319(8159303) WEXITSTATUS 0 Nov 04 23:42:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_319(8159303) done Nov 04 23:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_237(8159136) WEXITSTATUS 0 Nov 04 23:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_237(8159136) done Nov 04 23:42:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_255(8159154) WEXITSTATUS 0 Nov 04 23:42:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_255(8159154) done Nov 04 23:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_275(8159241) WEXITSTATUS 0 Nov 04 23:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_275(8159241) done Nov 04 23:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1263(8135181) WEXITSTATUS 0 Nov 04 23:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1263(8135181) done Nov 04 23:42:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5488(8160137) WEXITSTATUS 1 Nov 04 23:42:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5488(8160137) done Nov 04 23:43:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1975(8125411) Nov 04 23:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157484 WEXITSTATUS 0 Nov 04 23:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157484 done Nov 04 23:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_291(8159257) WEXITSTATUS 0 Nov 04 23:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_291(8159257) done Nov 04 23:43:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5422(8159467) WEXITSTATUS 1 Nov 04 23:43:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5422(8159467) done Nov 04 23:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157487 WEXITSTATUS 0 Nov 04 23:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157487 done Nov 04 23:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5122(8158462) WEXITSTATUS 1 Nov 04 23:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5122(8158462) done Nov 04 23:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157475 WEXITSTATUS 0 Nov 04 23:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157475 done Nov 04 23:43:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_253(8159152) WEXITSTATUS 0 Nov 04 23:43:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_253(8159152) done Nov 04 23:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_231(8159130) WEXITSTATUS 0 Nov 04 23:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_231(8159130) done Nov 04 23:43:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_235(8159134) WEXITSTATUS 0 Nov 04 23:43:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_235(8159134) done Nov 04 23:43:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_317(8159301) WEXITSTATUS 0 Nov 04 23:43:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_317(8159301) done Nov 04 23:43:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157733 NodeList=r816u15n12 #CPUs=12 Partition=scavenge Nov 04 23:43:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157734 NodeList=r816u29n08 #CPUs=12 Partition=scavenge Nov 04 23:43:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5131(8158471) WEXITSTATUS 1 Nov 04 23:43:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5131(8158471) done Nov 04 23:43:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157476 WEXITSTATUS 0 Nov 04 23:43:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157476 done Nov 04 23:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4968(8158230) WEXITSTATUS 1 Nov 04 23:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4968(8158230) done Nov 04 23:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_258(8159157) WEXITSTATUS 0 Nov 04 23:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_258(8159157) done Nov 04 23:43:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_264(8159230) WEXITSTATUS 0 Nov 04 23:43:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_264(8159230) done Nov 04 23:43:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_290(8159256) WEXITSTATUS 0 Nov 04 23:43:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_290(8159256) done Nov 04 23:43:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5259(8158693) WEXITSTATUS 1 Nov 04 23:43:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5259(8158693) done Nov 04 23:44:03 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_20(8154401) Nov 04 23:44:03 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_21(8154402) Nov 04 23:44:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 23:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_192(8159048) WEXITSTATUS 0 Nov 04 23:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_192(8159048) done Nov 04 23:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_249(8159148) WEXITSTATUS 0 Nov 04 23:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_249(8159148) done Nov 04 23:44:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156074 WEXITSTATUS 0 Nov 04 23:44:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156074 done Nov 04 23:44:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8160345 NodeList=(null) usec=17992 Nov 04 23:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160345 NodeList=r903u37n01 #CPUs=4 Partition=gpu_devel Nov 04 23:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157735 NodeList=r816u15n11 #CPUs=12 Partition=scavenge Nov 04 23:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8157736 NodeList=r816u23n08 #CPUs=12 Partition=scavenge Nov 04 23:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144118 NodeList=r801u27n03,r916u01n02 #CPUs=84 Partition=scavenge Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:44:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_230(8159129) WEXITSTATUS 0 Nov 04 23:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_230(8159129) done Nov 04 23:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157490 WEXITSTATUS 0 Nov 04 23:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157490 done Nov 04 23:44:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160346 InitPrio=17454 usec=3816 Nov 04 23:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_271(8159237) WEXITSTATUS 0 Nov 04 23:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_271(8159237) done Nov 04 23:44:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160346 NodeList=r801u23n02 #CPUs=4 Partition=pi_esi Nov 04 23:44:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160347 InitPrio=17454 usec=3640 Nov 04 23:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_274(7572442) WEXITSTATUS 0 Nov 04 23:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_274(7572442) done Nov 04 23:44:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160348 InitPrio=17454 usec=3184 Nov 04 23:44:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160347 NodeList=r801u04n04 #CPUs=4 Partition=pi_esi Nov 04 23:44:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1200(7572167) WEXITSTATUS 0 Nov 04 23:44:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1200(7572167) done Nov 04 23:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_326(8159310) WEXITSTATUS 0 Nov 04 23:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_326(8159310) done Nov 04 23:44:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160348 NodeList=r801u04n04 #CPUs=4 Partition=pi_esi Nov 04 23:44:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157485 WEXITSTATUS 0 Nov 04 23:44:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157485 done Nov 04 23:44:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_270(8159236) WEXITSTATUS 0 Nov 04 23:44:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_270(8159236) done Nov 04 23:45:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_309(8159293) WEXITSTATUS 0 Nov 04 23:45:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_309(8159293) done Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_278(8160349) in pi_ohern on r801u32n02 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_279(8160350) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_280(8160351) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_281(8160352) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_282(8160353) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_283(8160354) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_284(8160355) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_285(8160356) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_286(8160357) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_287(8160358) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_288(8160359) in pi_ohern on r917u13n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_289(8160360) in pi_ohern on r917u13n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_290(8160361) in pi_ohern on r917u13n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_291(8160362) in pi_ohern on r917u29n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_292(8160363) in pi_ohern on r917u29n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_293(8160364) in pi_ohern on r917u29n03 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_294(8160365) in pi_ohern on r917u29n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_295(8160366) in pi_ohern on r917u29n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_296(8160367) in pi_ohern on r917u29n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_297(8160368) in pi_ohern on r917u29n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_298(8160369) in pi_ohern on r917u29n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_299(8160370) in pi_ohern on r917u29n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_300(8160371) in pi_ohern on r917u29n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_301(8160372) in pi_ohern on r917u29n04 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_302(8160373) in pi_ohern on r815u03n11 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_303(8160374) in pi_ohern on r815u03n11 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_304(8160375) in pi_ohern on r815u03n11 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_305(8160376) in pi_ohern on r815u03n12 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_306(8160377) in pi_ohern on r903u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_307(8160378) in pi_ohern on r903u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_308(8160379) in pi_ohern on r903u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_309(8160380) in pi_ohern on r903u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_310(8160381) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_311(8160382) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_312(8160383) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_313(8160384) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_314(8160385) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_315(8160386) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_316(8160387) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_317(8160388) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_318(8160389) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_319(8160390) in pi_ohern on r913u35n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_320(8160391) in pi_ohern on r807u22n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_321(8160392) in pi_ohern on r807u22n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_322(8160393) in pi_ohern on r807u22n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_323(8160394) in pi_ohern on r807u22n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_324(8160395) in pi_ohern on r807u22n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_325(8160396) in pi_ohern on r807u22n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_326(8160397) in pi_ohern on r807u22n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_327(8160398) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_328(8160399) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_329(8160400) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_330(8160401) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_331(8160402) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_332(8160403) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_333(8160404) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_334(8160405) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_335(8160406) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_336(8160407) in pi_ohern on r807u24n01 Nov 04 23:45:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159904 in day on r815u29n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_200(8160408) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_201(8160409) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_202(8160410) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_203(8160411) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_204(8160412) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_205(8160413) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_206(8160414) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_207(8160415) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_208(8160416) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_209(8160417) in day on r807u11n04 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_210(8160418) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_211(8160419) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_212(8160420) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_213(8160421) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_214(8160422) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_215(8160423) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_216(8160424) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_217(8160425) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_218(8160426) in day on r904u10n01 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_219(8160427) in day on r806u02n03 Nov 04 23:45:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8150614 Nov 04 23:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_294(8159278) WEXITSTATUS 0 Nov 04 23:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_294(8159278) done Nov 04 23:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_368(8159377) WEXITSTATUS 0 Nov 04 23:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_368(8159377) done Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_220(8160428) in day on r806u02n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_221(8160429) in day on r806u02n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_222(8160430) in day on r806u02n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_223(8160431) in day on r806u02n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_224(8160432) in day on r806u02n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_225(8160433) in day on r806u02n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_226(8160434) in day on r806u02n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_227(8160435) in day on r806u02n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_228(8160436) in day on r806u11n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_229(8160437) in day on r806u11n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_230(8160438) in day on r806u11n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_231(8160439) in day on r806u11n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_232(8160440) in day on r806u11n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_233(8160441) in day on r806u11n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_234(8160442) in day on r806u14n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_235(8160443) in day on r806u14n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_236(8160444) in day on r806u14n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_237(8160445) in day on r806u14n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_238(8160446) in day on r806u14n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_239(8160447) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_240(8160448) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_241(8160449) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_242(8160450) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_243(8160451) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_244(8160452) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_245(8160453) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_246(8160454) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_247(8160455) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_248(8160456) in day on r806u14n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_249(8160457) in day on r806u14n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_250(8160458) in day on r806u14n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_251(8160459) in day on r806u14n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_252(8160460) in day on r806u14n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_253(8160461) in day on r806u14n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_254(8160462) in day on r806u14n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_255(8160463) in day on r806u14n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_256(8160464) in day on r909u01n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_257(8160465) in day on r909u01n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_258(8160466) in day on r909u01n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_259(8160467) in day on r909u01n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_260(8160468) in day on r909u01n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_261(8160469) in day on r909u01n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_262(8160470) in day on r909u01n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_263(8160471) in day on r909u01n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_264(8160472) in day on r909u01n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_265(8160473) in day on r909u04n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_266(8160474) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_267(8160475) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_268(8160476) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_269(8160477) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_270(8160478) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_271(8160479) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_272(8160480) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_273(8160481) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_274(8160482) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_275(8160483) in day on r909u04n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_276(8160484) in day on r909u04n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_277(8160485) in day on r909u04n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_278(8160486) in day on r909u04n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_279(8160487) in day on r909u04n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_280(8160488) in day on r909u04n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_281(8160489) in day on r909u04n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_282(8160490) in day on r909u04n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_283(8160491) in day on r909u04n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_284(8160492) in day on r909u04n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_285(8160493) in day on r909u07n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_286(8160494) in day on r909u07n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_287(8160495) in day on r909u07n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_288(8160496) in day on r909u07n02 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_289(8160497) in day on r909u07n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_290(8160498) in day on r909u07n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_291(8160499) in day on r909u07n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_292(8160500) in day on r909u07n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_293(8160501) in day on r909u07n03 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_294(8160502) in day on r909u07n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_295(8160503) in day on r909u07n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_296(8160504) in day on r909u07n04 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_297(8160505) in day on r909u10n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_298(8160506) in day on r909u10n01 Nov 04 23:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_299(8160507) in day on r909u10n01 Nov 04 23:45:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r914u01n02 Nov 04 23:45:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_330(8159314) WEXITSTATUS 0 Nov 04 23:45:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_330(8159314) done Nov 04 23:45:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3581(8159317) WEXITSTATUS 0 Nov 04 23:45:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3581(8159317) done Nov 04 23:45:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_269(8159235) WEXITSTATUS 0 Nov 04 23:45:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_269(8159235) done Nov 04 23:45:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_236(8159135) WEXITSTATUS 0 Nov 04 23:45:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_236(8159135) done Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5505(8160508) in scavenge on r907u01n03 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5506(8160509) in scavenge on r907u05n01 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5507(8160510) in scavenge on r802u06n03 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5508(8160511) in scavenge on r802u25n01 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5509(8160512) in scavenge on r815u15n12 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5510(8160513) in scavenge on r816u23n08 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5511(8160514) in scavenge on r909u13n02 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5512(8160515) in scavenge on r919u09n01 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5513(8160516) in scavenge on r815u09n01 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5514(8160517) in scavenge on r815u09n01 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5515(8160518) in scavenge on r815u09n01 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5516(8160519) in scavenge on r816u09n05 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_261(8159227) WEXITSTATUS 0 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_261(8159227) done Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4339(8154949) WEXITSTATUS 1 Nov 04 23:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4339(8154949) done Nov 04 23:45:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_21(8160520) in scavenge on r815u03n10 Nov 04 23:45:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_22(8160521) in scavenge on r815u03n10 Nov 04 23:45:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_262(8159228) WEXITSTATUS 0 Nov 04 23:45:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_262(8159228) done Nov 04 23:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5071(8158383) WEXITSTATUS 1 Nov 04 23:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5071(8158383) done Nov 04 23:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_332(8159316) WEXITSTATUS 0 Nov 04 23:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_332(8159316) done Nov 04 23:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150057 WEXITSTATUS 0 Nov 04 23:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150057 done Nov 04 23:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5212(8158599) WEXITSTATUS 1 Nov 04 23:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5212(8158599) done Nov 04 23:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_320(8159304) WEXITSTATUS 0 Nov 04 23:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_320(8159304) done Nov 04 23:45:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1511(8143570) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:45:34 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157491 had to be killed Nov 04 23:45:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157482 WEXITSTATUS 0 Nov 04 23:45:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157482 done Nov 04 23:45:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1861(8160522) in scavenge on r816u23n08 Nov 04 23:45:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1862(8160523) in scavenge on r816u23n08 Nov 04 23:45:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1511(8143570) Nov 04 23:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_406(8159433) WEXITSTATUS 0 Nov 04 23:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_406(8159433) done Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:45:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_232(8159131) WEXITSTATUS 0 Nov 04 23:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_232(8159131) done Nov 04 23:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_367(8159376) WEXITSTATUS 0 Nov 04 23:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_367(8159376) done Nov 04 23:45:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_321(8159305) WEXITSTATUS 0 Nov 04 23:45:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_321(8159305) done Nov 04 23:45:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_268(8159234) WEXITSTATUS 0 Nov 04 23:45:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_268(8159234) done Nov 04 23:45:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5224(8158611) WEXITSTATUS 1 Nov 04 23:45:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5224(8158611) done Nov 04 23:45:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_366(8159375) WEXITSTATUS 0 Nov 04 23:45:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_366(8159375) done Nov 04 23:46:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4314(8154197) WEXITSTATUS 1 Nov 04 23:46:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4314(8154197) done Nov 04 23:46:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5453(8159769) WEXITSTATUS 1 Nov 04 23:46:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5453(8159769) done Nov 04 23:46:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160346 WEXITSTATUS 0 Nov 04 23:46:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160346 done Nov 04 23:46:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_272(8159238) WEXITSTATUS 0 Nov 04 23:46:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_272(8159238) done Nov 04 23:46:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_27(8135726) WEXITSTATUS 0 Nov 04 23:46:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_27(8135726) done Nov 04 23:46:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1266(8135193) WEXITSTATUS 0 Nov 04 23:46:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1266(8135193) done Nov 04 23:46:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157488 WEXITSTATUS 0 Nov 04 23:46:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157488 done Nov 04 23:46:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5294(8158742) WEXITSTATUS 1 Nov 04 23:46:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5294(8158742) done Nov 04 23:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157489 WEXITSTATUS 0 Nov 04 23:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157489 done Nov 04 23:46:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_422(8159449) WEXITSTATUS 0 Nov 04 23:46:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_422(8159449) done Nov 04 23:46:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_297(8159281) WEXITSTATUS 0 Nov 04 23:46:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_297(8159281) done Nov 04 23:46:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:46:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5040(8158336) WEXITSTATUS 1 Nov 04 23:46:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5040(8158336) done Nov 04 23:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160347 WEXITSTATUS 0 Nov 04 23:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160347 done Nov 04 23:46:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_325(8159309) WEXITSTATUS 0 Nov 04 23:46:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_325(8159309) done Nov 04 23:46:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4907(8158057) WEXITSTATUS 1 Nov 04 23:46:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4907(8158057) done Nov 04 23:46:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160348 WEXITSTATUS 0 Nov 04 23:46:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160348 done Nov 04 23:46:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_299(8159283) WEXITSTATUS 0 Nov 04 23:46:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_299(8159283) done Nov 04 23:46:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_323(8159307) WEXITSTATUS 0 Nov 04 23:46:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_323(8159307) done Nov 04 23:46:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_273(8159239) WEXITSTATUS 0 Nov 04 23:46:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_273(8159239) done Nov 04 23:47:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_265(8159231) WEXITSTATUS 0 Nov 04 23:47:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_265(8159231) done Nov 04 23:47:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160524 InitPrio=17454 usec=3533 Nov 04 23:47:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160524 NodeList=r801u04n04 #CPUs=4 Partition=pi_esi Nov 04 23:47:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160525 InitPrio=17454 usec=2295 Nov 04 23:47:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160526 InitPrio=17454 usec=3739 Nov 04 23:47:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160527 InitPrio=17454 usec=3469 Nov 04 23:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160525 NodeList=r801u04n04 #CPUs=4 Partition=pi_esi Nov 04 23:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160526 NodeList=r801u23n02 #CPUs=4 Partition=pi_esi Nov 04 23:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_674(8114943) WEXITSTATUS 0 Nov 04 23:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_674(8114943) done Nov 04 23:47:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160528 InitPrio=17454 usec=5198 Nov 04 23:47:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160529 InitPrio=17454 usec=4902 Nov 04 23:47:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160527 NodeList=r801u11n02 #CPUs=4 Partition=pi_esi Nov 04 23:47:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160530 InitPrio=17454 usec=3502 Nov 04 23:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_304(8159288) WEXITSTATUS 0 Nov 04 23:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_304(8159288) done Nov 04 23:47:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_327(8159311) WEXITSTATUS 0 Nov 04 23:47:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_327(8159311) done Nov 04 23:47:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160531 InitPrio=17454 usec=3303 Nov 04 23:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_328(8159312) WEXITSTATUS 0 Nov 04 23:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_328(8159312) done Nov 04 23:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_300(8159284) WEXITSTATUS 0 Nov 04 23:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_300(8159284) done Nov 04 23:47:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_267(8159233) WEXITSTATUS 0 Nov 04 23:47:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_267(8159233) done Nov 04 23:47:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_260(8159226) WEXITSTATUS 0 Nov 04 23:47:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_260(8159226) done Nov 04 23:47:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160532 InitPrio=17454 usec=3524 Nov 04 23:47:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160533 InitPrio=17454 usec=3747 Nov 04 23:47:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3607(8160327) WEXITSTATUS 0 Nov 04 23:47:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3607(8160327) done Nov 04 23:47:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160534 InitPrio=17454 usec=3228 Nov 04 23:47:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160535 InitPrio=17454 usec=3383 Nov 04 23:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_369(8159378) WEXITSTATUS 0 Nov 04 23:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_369(8159378) done Nov 04 23:47:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160536 InitPrio=34646 usec=5781 Nov 04 23:47:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160537 InitPrio=34646 usec=5502 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_396(8159423) WEXITSTATUS 0 Nov 04 23:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_396(8159423) done Nov 04 23:47:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4936(8158145) WEXITSTATUS 1 Nov 04 23:47:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4936(8158145) done Nov 04 23:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_394(8159421) WEXITSTATUS 0 Nov 04 23:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_394(8159421) done Nov 04 23:47:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_301(8159285) WEXITSTATUS 0 Nov 04 23:47:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_301(8159285) done Nov 04 23:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_419(8159446) WEXITSTATUS 0 Nov 04 23:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_419(8159446) done Nov 04 23:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_274(8159240) WEXITSTATUS 0 Nov 04 23:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_274(8159240) done Nov 04 23:47:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8160538 InitPrio=36030 usec=4816 Nov 04 23:48:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_365(8159374) WEXITSTATUS 0 Nov 04 23:48:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_365(8159374) done Nov 04 23:48:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2144(8125964) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:48:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2060(8151620) WEXITSTATUS 0 Nov 04 23:48:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2060(8151620) done Nov 04 23:48:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2144(8125964) Nov 04 23:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_302(8159286) WEXITSTATUS 0 Nov 04 23:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_302(8159286) done Nov 04 23:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_266(8159232) WEXITSTATUS 0 Nov 04 23:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_266(8159232) done Nov 04 23:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160528 NodeList=r801u25n01 #CPUs=4 Partition=pi_esi Nov 04 23:48:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8127833 WEXITSTATUS 255 Nov 04 23:48:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8127833 done Nov 04 23:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_315(8159299) WEXITSTATUS 0 Nov 04 23:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_315(8159299) done Nov 04 23:48:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160529 NodeList=r801u04n02 #CPUs=4 Partition=pi_esi Nov 04 23:48:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160530 NodeList=r801u04n02 #CPUs=4 Partition=pi_esi Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_337(8160539) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_338(8160540) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_339(8160541) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_340(8160542) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_341(8160543) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_342(8160544) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_343(8160545) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_344(8160546) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_345(8160547) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_346(8160548) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_347(8160549) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_348(8160550) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_349(8160551) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_350(8160552) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_351(8160553) in pi_ohern on r917u13n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_352(8160554) in pi_ohern on r917u29n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_353(8160555) in pi_ohern on r917u29n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_354(8160556) in pi_ohern on r917u29n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_355(8160557) in pi_ohern on r917u29n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_356(8160558) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_357(8160559) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_358(8160560) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_359(8160561) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_360(8160562) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_361(8160563) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_362(8160564) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_363(8160565) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_364(8160566) in pi_ohern on r917u29n03 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_365(8160567) in pi_ohern on r917u29n04 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_366(8160568) in pi_ohern on r917u29n04 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_367(8160569) in pi_ohern on r917u29n04 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_368(8160570) in pi_ohern on r917u29n04 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_369(8160571) in pi_ohern on r917u29n04 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_370(8160572) in pi_ohern on r917u29n04 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_371(8160573) in pi_ohern on r917u29n04 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_372(8160574) in pi_ohern on r917u29n04 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_373(8160575) in pi_ohern on r815u03n11 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_374(8160576) in pi_ohern on r815u03n11 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_375(8160577) in pi_ohern on r815u03n11 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_376(8160578) in pi_ohern on r815u03n11 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_377(8160579) in pi_ohern on r815u03n11 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_378(8160580) in pi_ohern on r815u03n11 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_379(8160581) in pi_ohern on r815u03n12 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_380(8160582) in pi_ohern on r815u03n12 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_381(8160583) in pi_ohern on r815u03n12 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_382(8160584) in pi_ohern on r815u03n12 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_383(8160585) in pi_ohern on r815u03n12 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_384(8160586) in pi_ohern on r815u03n12 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_385(8160587) in pi_ohern on r807u22n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_386(8160588) in pi_ohern on r807u22n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_387(8160589) in pi_ohern on r807u22n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_388(8160590) in pi_ohern on r807u22n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_389(8160591) in pi_ohern on r807u22n01 Nov 04 23:48:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_390(8160592) in pi_ohern on r807u22n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8160538 in day on r807u05n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8160536 in day on r914u01n02 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8160537 in day on r914u05n03 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159886 in day on r914u05n03 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159906 in day on r815u15n08 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_300(8160593) in day on r807u05n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_301(8160594) in day on r807u05n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_302(8160595) in day on r914u05n03 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_303(8160596) in day on r909u10n02 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_304(8160597) in day on r909u10n03 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_305(8160598) in day on r909u10n04 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_306(8160599) in day on r909u13n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_307(8160600) in day on r909u13n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_308(8160601) in day on r909u13n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_309(8160602) in day on r909u13n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_310(8160603) in day on r909u13n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_311(8160604) in day on r909u13n01 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_312(8160605) in day on r909u13n02 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_313(8160606) in day on r909u13n02 Nov 04 23:48:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_314(8160607) in day on r909u13n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_315(8160608) in day on r909u13n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_316(8160609) in day on r909u13n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_317(8160610) in day on r909u13n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_318(8160611) in day on r909u13n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_319(8160612) in day on r909u13n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_320(8160613) in day on r909u13n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_321(8160614) in day on r909u13n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_322(8160615) in day on r909u13n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_323(8160616) in day on r909u13n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_324(8160617) in day on r909u29n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_325(8160618) in day on r909u29n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_326(8160619) in day on r909u29n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_327(8160620) in day on r909u29n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_328(8160621) in day on r909u29n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_329(8160622) in day on r909u29n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_330(8160623) in day on r909u29n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_331(8160624) in day on r909u29n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_332(8160625) in day on r909u29n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_333(8160626) in day on r912u28n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_334(8160627) in day on r912u28n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_335(8160628) in day on r912u28n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_336(8160629) in day on r912u28n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_337(8160630) in day on r912u28n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_338(8160631) in day on r912u28n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_339(8160632) in day on r912u28n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_340(8160633) in day on r912u28n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_341(8160634) in day on r912u28n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_342(8160635) in day on r912u28n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_343(8160636) in day on r912u28n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_344(8160637) in day on r912u28n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_345(8160638) in day on r912u32n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_346(8160639) in day on r912u32n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_347(8160640) in day on r912u32n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_348(8160641) in day on r912u32n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_349(8160642) in day on r912u32n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_350(8160643) in day on r912u32n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_351(8160644) in day on r912u32n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_352(8160645) in day on r912u32n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_353(8160646) in day on r912u32n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_354(8160647) in day on r912u32n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_355(8160648) in day on r912u32n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_356(8160649) in day on r912u36n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_357(8160650) in day on r912u36n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_358(8160651) in day on r912u36n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_359(8160652) in day on r912u36n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_360(8160653) in day on r912u36n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_361(8160654) in day on r912u36n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_362(8160655) in day on r912u36n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_363(8160656) in day on r912u36n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_364(8160657) in day on r912u36n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_365(8160658) in day on r912u36n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_366(8160659) in day on r912u36n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_367(8160660) in day on r912u36n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:146 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_368(8160661) in day on r912u36n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_369(8160662) in day on r912u36n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_370(8160663) in day on r912u36n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_371(8160664) in day on r912u36n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_372(8160665) in day on r912u36n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_373(8160666) in day on r913u01n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_374(8160667) in day on r913u01n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_375(8160668) in day on r913u01n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_376(8160669) in day on r913u01n02 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_377(8160670) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_378(8160671) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_379(8160672) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_380(8160673) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_381(8160674) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_382(8160675) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_383(8160676) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_384(8160677) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_385(8160678) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_386(8160679) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_387(8160680) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_388(8160681) in day on r913u01n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_389(8160682) in day on r913u01n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_390(8160683) in day on r913u01n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_391(8160684) in day on r913u01n04 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_392(8160685) in day on r913u05n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_393(8160686) in day on r913u05n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_394(8160687) in day on r913u05n01 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_395(8160688) in day on r913u05n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_396(8160689) in day on r913u05n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_397(8160690) in day on r913u05n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_398(8160691) in day on r913u05n03 Nov 04 23:48:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_399(8160692) in day on r913u05n04 Nov 04 23:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3609(8160693) in gpu on r808u17n01 Nov 04 23:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3610(8160694) in gpu on r903u40n01 Nov 04 23:48:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 04 23:48:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1863(8160695) in day on r815u29n01 Nov 04 23:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3610(8160694) WEXITSTATUS 0 Nov 04 23:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3610(8160694) done Nov 04 23:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_409(8159436) WEXITSTATUS 0 Nov 04 23:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_409(8159436) done Nov 04 23:48:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_348(8159357) WEXITSTATUS 0 Nov 04 23:48:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_348(8159357) done Nov 04 23:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1831(8151157) WEXITSTATUS 0 Nov 04 23:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1831(8151157) done Nov 04 23:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_329(8159313) WEXITSTATUS 0 Nov 04 23:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_329(8159313) done Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5517(8160696) in scavenge on r802u04n04 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5518(8160697) in scavenge on r802u14n02 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5519(8160698) in scavenge on r816u23n10 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5520(8160699) in scavenge on r816u15n10 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5521(8160700) in scavenge on r816u15n12 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5522(8160701) in scavenge on r907u32n01 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5523(8160702) in scavenge on r907u34n01 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5524(8160703) in scavenge on r906u28n01 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5525(8160704) in scavenge on r805u02n02 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5526(8160705) in scavenge on r918u05n02 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5527(8160706) in scavenge on r918u32n02 Nov 04 23:48:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5528(8160707) in scavenge on r815u09n01 Nov 04 23:48:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5264(8158698) WEXITSTATUS 1 Nov 04 23:48:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5264(8158698) done Nov 04 23:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_407(8159434) WEXITSTATUS 0 Nov 04 23:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_407(8159434) done Nov 04 23:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_415(8159442) WEXITSTATUS 0 Nov 04 23:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_415(8159442) done Nov 04 23:48:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5272(8158706) WEXITSTATUS 1 Nov 04 23:48:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5272(8158706) done Nov 04 23:48:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1493(8140412) WEXITSTATUS 0 Nov 04 23:48:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1493(8140412) done Nov 04 23:48:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1410(8150142) WEXITSTATUS 0 Nov 04 23:48:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1410(8150142) done Nov 04 23:48:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157494 WEXITSTATUS 0 Nov 04 23:48:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157494 done Nov 04 23:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2313(8152230) WEXITSTATUS 0 Nov 04 23:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2313(8152230) done Nov 04 23:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157483 WEXITSTATUS 0 Nov 04 23:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157483 done Nov 04 23:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_324(8159308) WEXITSTATUS 0 Nov 04 23:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_324(8159308) done Nov 04 23:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4374(8155201) WEXITSTATUS 1 Nov 04 23:48:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4374(8155201) done Nov 04 23:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_400(8159427) WEXITSTATUS 0 Nov 04 23:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_400(8159427) done Nov 04 23:48:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_417(8159444) WEXITSTATUS 0 Nov 04 23:48:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_417(8159444) done Nov 04 23:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_30(8158662) WEXITSTATUS 0 Nov 04 23:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_30(8158662) done Nov 04 23:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_408(8159435) WEXITSTATUS 0 Nov 04 23:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_408(8159435) done Nov 04 23:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4708(8156591) WEXITSTATUS 1 Nov 04 23:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4708(8156591) done Nov 04 23:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_298(8159282) WEXITSTATUS 0 Nov 04 23:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_298(8159282) done Nov 04 23:49:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_860(8127974) has been requeued to reclaim resources for JobId=8160531 Nov 04 23:49:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_860(8127974) Nov 04 23:49:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160531 NodeList=r801u14n03 #CPUs=4 Partition=pi_esi Nov 04 23:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_303(8159287) WEXITSTATUS 0 Nov 04 23:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_303(8159287) done Nov 04 23:49:06 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1512(8143571) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4857(8157942) WEXITSTATUS 1 Nov 04 23:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4857(8157942) done Nov 04 23:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_311(8159295) WEXITSTATUS 0 Nov 04 23:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_311(8159295) done Nov 04 23:49:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1512(8143571) Nov 04 23:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_401(8159428) WEXITSTATUS 0 Nov 04 23:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_401(8159428) done Nov 04 23:49:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_337(8159346) WEXITSTATUS 0 Nov 04 23:49:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_337(8159346) done Nov 04 23:49:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_402(8159429) WEXITSTATUS 0 Nov 04 23:49:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_402(8159429) done Nov 04 23:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_296(8159280) WEXITSTATUS 0 Nov 04 23:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_296(8159280) done Nov 04 23:49:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_403(8159430) WEXITSTATUS 0 Nov 04 23:49:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_403(8159430) done Nov 04 23:49:35 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2407(8126446) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:49:37 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2407(8126446) Nov 04 23:49:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160532 NodeList=r801u14n03 #CPUs=4 Partition=pi_esi Nov 04 23:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_331(8159315) WEXITSTATUS 0 Nov 04 23:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_331(8159315) done Nov 04 23:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_306(8159290) WEXITSTATUS 0 Nov 04 23:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_306(8159290) done Nov 04 23:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_398(8159425) WEXITSTATUS 0 Nov 04 23:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_398(8159425) done Nov 04 23:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_346(8159355) WEXITSTATUS 0 Nov 04 23:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_346(8159355) done Nov 04 23:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_414(8159441) WEXITSTATUS 0 Nov 04 23:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_414(8159441) done Nov 04 23:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_345(8159354) WEXITSTATUS 0 Nov 04 23:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_345(8159354) done Nov 04 23:49:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_418(8159445) WEXITSTATUS 0 Nov 04 23:49:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_418(8159445) done Nov 04 23:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_416(8159443) WEXITSTATUS 0 Nov 04 23:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_416(8159443) done Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_410(8159437) WEXITSTATUS 0 Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_410(8159437) done Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_483(8159525) WEXITSTATUS 0 Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_483(8159525) done Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_293(8159277) WEXITSTATUS 0 Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_293(8159277) done Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5491(8160140) WEXITSTATUS 1 Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5491(8160140) done Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_393(8159420) WEXITSTATUS 0 Nov 04 23:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_393(8159420) done Nov 04 23:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2392(8126421) WEXITSTATUS 1 Nov 04 23:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2392(8126421) done Nov 04 23:49:50 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:49:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:49:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4997(8158276) WEXITSTATUS 1 Nov 04 23:49:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4997(8158276) done Nov 04 23:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5523(8160702) WEXITSTATUS 1 Nov 04 23:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5523(8160702) done Nov 04 23:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_314(8159298) WEXITSTATUS 0 Nov 04 23:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_314(8159298) done Nov 04 23:49:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_310(8159294) WEXITSTATUS 0 Nov 04 23:49:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_310(8159294) done Nov 04 23:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_405(8159432) WEXITSTATUS 0 Nov 04 23:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_405(8159432) done Nov 04 23:50:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_33(8158665) WEXITSTATUS 0 Nov 04 23:50:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_33(8158665) done Nov 04 23:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_397(8159424) WEXITSTATUS 0 Nov 04 23:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_397(8159424) done Nov 04 23:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_395(8159422) WEXITSTATUS 0 Nov 04 23:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_395(8159422) done Nov 04 23:50:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_307(8159291) WEXITSTATUS 0 Nov 04 23:50:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_307(8159291) done Nov 04 23:50:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_353(8159362) WEXITSTATUS 0 Nov 04 23:50:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_353(8159362) done Nov 04 23:50:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1120(8131612) has been requeued to reclaim resources for JobId=8160533 Nov 04 23:50:32 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1120(8131612) Nov 04 23:50:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160533 NodeList=r801u06n01 #CPUs=4 Partition=pi_esi Nov 04 23:50:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4850(8157875) WEXITSTATUS 1 Nov 04 23:50:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4850(8157875) done Nov 04 23:50:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_344(8159353) WEXITSTATUS 0 Nov 04 23:50:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_344(8159353) done Nov 04 23:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_349(8159358) WEXITSTATUS 0 Nov 04 23:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_349(8159358) done Nov 04 23:50:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_308(8159292) WEXITSTATUS 0 Nov 04 23:50:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_308(8159292) done Nov 04 23:50:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_313(8159297) WEXITSTATUS 0 Nov 04 23:50:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_313(8159297) done Nov 04 23:50:43 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8160708 NodeList=(null) usec=24230 Nov 04 23:50:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4376(8155203) WEXITSTATUS 1 Nov 04 23:50:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4376(8155203) done Nov 04 23:50:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160708 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 04 23:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_471(8159513) WEXITSTATUS 0 Nov 04 23:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_471(8159513) done Nov 04 23:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_484(8159526) WEXITSTATUS 0 Nov 04 23:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_484(8159526) done Nov 04 23:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2123(8151799) WEXITSTATUS 0 Nov 04 23:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2123(8151799) done Nov 04 23:50:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_399(8159426) WEXITSTATUS 0 Nov 04 23:50:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_399(8159426) done Nov 04 23:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2200(8151992) WEXITSTATUS 0 Nov 04 23:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2200(8151992) done Nov 04 23:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5499(8160336) WEXITSTATUS 1 Nov 04 23:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5499(8160336) done Nov 04 23:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2141(8151817) WEXITSTATUS 0 Nov 04 23:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2141(8151817) done Nov 04 23:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4864(8157963) WEXITSTATUS 1 Nov 04 23:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4864(8157963) done Nov 04 23:51:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_333(8159342) WEXITSTATUS 0 Nov 04 23:51:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_333(8159342) done Nov 04 23:51:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_1(8158960) WEXITSTATUS 0 Nov 04 23:51:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_1(8158960) done Nov 04 23:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_424(8159451) WEXITSTATUS 0 Nov 04 23:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_424(8159451) done Nov 04 23:51:06 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1436(8135969) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:51:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8151041 Nov 04 23:51:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_22(8154972) Nov 04 23:51:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_23(8154973) Nov 04 23:51:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_24(8154974) Nov 04 23:51:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_25(8154975) Nov 04 23:51:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4552(8155961) WEXITSTATUS 1 Nov 04 23:51:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4552(8155961) done Nov 04 23:51:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1436(8135969) Nov 04 23:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2171(8151963) WEXITSTATUS 0 Nov 04 23:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2171(8151963) done Nov 04 23:51:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160534 NodeList=r801u06n02 #CPUs=4 Partition=pi_esi Nov 04 23:51:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_347(8159356) WEXITSTATUS 0 Nov 04 23:51:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_347(8159356) done Nov 04 23:51:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_404(8159431) WEXITSTATUS 0 Nov 04 23:51:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_404(8159431) done Nov 04 23:51:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5334(8158953) WEXITSTATUS 1 Nov 04 23:51:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5334(8158953) done Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157493 WEXITSTATUS 0 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157493 done Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_391(8160709) in pi_ohern on r917u13n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_392(8160710) in pi_ohern on r917u13n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_393(8160711) in pi_ohern on r917u13n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_394(8160712) in pi_ohern on r917u29n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_395(8160713) in pi_ohern on r917u29n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_396(8160714) in pi_ohern on r917u29n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_397(8160715) in pi_ohern on r917u29n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_398(8160716) in pi_ohern on r917u29n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_399(8160717) in pi_ohern on r917u29n02 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_400(8160718) in pi_ohern on r917u29n02 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_401(8160719) in pi_ohern on r917u29n02 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_402(8160720) in pi_ohern on r917u29n02 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_403(8160721) in pi_ohern on r917u29n02 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_404(8160722) in pi_ohern on r917u29n03 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_405(8160723) in pi_ohern on r917u29n03 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_406(8160724) in pi_ohern on r917u29n03 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_407(8160725) in pi_ohern on r917u29n03 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_408(8160726) in pi_ohern on r917u29n03 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_409(8160727) in pi_ohern on r917u29n03 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_410(8160728) in pi_ohern on r917u29n04 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_411(8160729) in pi_ohern on r917u29n04 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_412(8160730) in pi_ohern on r917u29n04 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_413(8160731) in pi_ohern on r917u29n04 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_414(8160732) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_415(8160733) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_416(8160734) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_417(8160735) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_418(8160736) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_419(8160737) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_420(8160738) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_421(8160739) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_422(8160740) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_423(8160741) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_424(8160742) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_425(8160743) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_426(8160744) in pi_ohern on r815u03n11 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_427(8160745) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_428(8160746) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_429(8160747) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_430(8160748) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_431(8160749) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_432(8160750) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_433(8160751) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_434(8160752) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_435(8160753) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_436(8160754) in pi_ohern on r815u03n12 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_437(8160755) in pi_ohern on r807u22n01 Nov 04 23:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_438(8160756) in pi_ohern on r807u22n01 Nov 04 23:51:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_439(8160757) in pi_ohern on r807u22n01 Nov 04 23:51:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_440(8160758) in pi_ohern on r911u32n01 Nov 04 23:51:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159907 in day on r816u03n07 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_400(8160759) in day on r807u03n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_401(8160760) in day on r909u07n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_402(8160761) in day on r909u07n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_403(8160762) in day on r909u07n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_404(8160763) in day on r909u07n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_405(8160764) in day on r909u07n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_406(8160765) in day on r806u02n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_407(8160766) in day on r806u14n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_408(8160767) in day on r914u05n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_409(8160768) in day on r914u05n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_410(8160769) in day on r914u05n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_411(8160770) in day on r914u05n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_412(8160771) in day on r912u36n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_413(8160772) in day on r913u05n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_414(8160773) in day on r913u05n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_415(8160774) in day on r913u05n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_416(8160775) in day on r913u09n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_417(8160776) in day on r913u09n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_418(8160777) in day on r913u09n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_419(8160778) in day on r913u09n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_420(8160779) in day on r913u09n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_421(8160780) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_422(8160781) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_423(8160782) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_424(8160783) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_425(8160784) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_426(8160785) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_427(8160786) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_428(8160787) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_429(8160788) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_430(8160789) in day on r913u09n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_431(8160790) in day on r913u13n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_432(8160791) in day on r913u13n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_433(8160792) in day on r913u13n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_434(8160793) in day on r913u13n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_435(8160794) in day on r913u24n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_436(8160795) in day on r913u24n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_437(8160796) in day on r913u24n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_438(8160797) in day on r913u24n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_439(8160798) in day on r913u24n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_440(8160799) in day on r913u24n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_441(8160800) in day on r913u24n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_442(8160801) in day on r913u24n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_443(8160802) in day on r913u24n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_444(8160803) in day on r913u24n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_445(8160804) in day on r913u28n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_446(8160805) in day on r913u28n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_447(8160806) in day on r913u28n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_448(8160807) in day on r913u28n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_449(8160808) in day on r913u28n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_450(8160809) in day on r913u28n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_451(8160810) in day on r913u28n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_452(8160811) in day on r913u28n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_453(8160812) in day on r913u28n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_454(8160813) in day on r913u28n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_455(8160814) in day on r913u28n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_456(8160815) in day on r913u28n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_457(8160816) in day on r913u28n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_458(8160817) in day on r913u28n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_425(8159452) WEXITSTATUS 0 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_425(8159452) done Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_459(8160818) in day on r913u28n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_460(8160819) in day on r913u28n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_461(8160820) in day on r913u28n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_462(8160821) in day on r805u02n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_463(8160822) in day on r805u02n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_464(8160823) in day on r805u02n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_465(8160824) in day on r805u02n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_466(8160825) in day on r805u02n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_467(8160826) in day on r805u02n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_468(8160827) in day on r805u02n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_469(8160828) in day on r805u02n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_470(8160829) in day on r805u02n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_471(8160830) in day on r805u02n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_472(8160831) in day on r805u02n04 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_473(8160832) in day on r805u04n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_474(8160833) in day on r805u04n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_475(8160834) in day on r805u04n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_476(8160835) in day on r805u04n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_477(8160836) in day on r805u04n01 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_478(8160837) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_479(8160838) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_480(8160839) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_481(8160840) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_482(8160841) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_483(8160842) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_484(8160843) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_485(8160844) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_486(8160845) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_487(8160846) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_488(8160847) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_489(8160848) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_490(8160849) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_491(8160850) in day on r805u04n02 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_492(8160851) in day on r805u04n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_493(8160852) in day on r805u04n03 Nov 04 23:51:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_494(8160853) in day on r805u04n03 Nov 04 23:51:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_495(8160854) in day on r805u04n03 Nov 04 23:51:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_496(8160855) in day on r805u04n04 Nov 04 23:51:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_497(8160856) in day on r805u04n04 Nov 04 23:51:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_498(8160857) in day on r805u04n04 Nov 04 23:51:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_499(8160858) in day on r805u04n04 Nov 04 23:51:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_23(8160859) in day on r815u03n08 Nov 04 23:51:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_24(8160860) in day on r815u09n02 Nov 04 23:51:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3611(8160861) in gpu on r903u40n01 Nov 04 23:51:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1864(8160862) in day on r816u03n07 Nov 04 23:51:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1865(8160863) in day on r816u03n07 Nov 04 23:51:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r909u07n01 Nov 04 23:51:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_412(8159439) WEXITSTATUS 0 Nov 04 23:51:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_412(8159439) done Nov 04 23:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_426(8159453) WEXITSTATUS 0 Nov 04 23:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_426(8159453) done Nov 04 23:51:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_263(8159229) WEXITSTATUS 0 Nov 04 23:51:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_263(8159229) done Nov 04 23:51:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5050(8158346) WEXITSTATUS 1 Nov 04 23:51:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5050(8158346) done Nov 04 23:51:36 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158751 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5529(8160864) in scavenge on r801u23n03 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5530(8160865) in scavenge on r903u01n01 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5531(8160866) in scavenge on r917u01n02 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5532(8160867) in scavenge on r917u10n03 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5533(8160868) in scavenge on r815u03n09 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5534(8160869) in scavenge on r816u03n12 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5535(8160870) in scavenge on r816u29n08 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5536(8160871) in scavenge on r816u29n11 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5537(8160872) in scavenge on r816u29n11 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5538(8160873) in scavenge on r907u34n01 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5539(8160874) in scavenge on r805u06n03 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5540(8160875) in scavenge on r918u36n02 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5541(8160876) in scavenge on r919u01n01 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5542(8160877) in scavenge on r919u05n03 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5543(8160878) in scavenge on r919u36n01 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5544(8160879) in scavenge on r919u36n01 Nov 04 23:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5545(8160880) in scavenge on r816u03n07 Nov 04 23:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_184(7572352) in scavenge on r816u09n10 Nov 04 23:51:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_354(8159363) WEXITSTATUS 0 Nov 04 23:51:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_354(8159363) done Nov 04 23:51:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_411(8159438) WEXITSTATUS 0 Nov 04 23:51:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_411(8159438) done Nov 04 23:51:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_312(8159296) WEXITSTATUS 0 Nov 04 23:51:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_312(8159296) done Nov 04 23:51:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_413(8159440) WEXITSTATUS 0 Nov 04 23:51:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_413(8159440) done Nov 04 23:51:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_350(8159359) WEXITSTATUS 0 Nov 04 23:51:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_350(8159359) done Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1462(8138139) in scavenge on r816u29n08 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1402(8135935) in scavenge on r816u29n08 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1463(8138140) in scavenge on r816u29n11 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1403(8135936) in scavenge on r816u29n11 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1460(8138137) in scavenge on r816u29n11 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:51:50 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:51:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_364(8159373) WEXITSTATUS 0 Nov 04 23:51:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_364(8159373) done Nov 04 23:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_339(8159348) WEXITSTATUS 0 Nov 04 23:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_339(8159348) done Nov 04 23:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5232(8158635) WEXITSTATUS 1 Nov 04 23:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5232(8158635) done Nov 04 23:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_341(8159350) WEXITSTATUS 0 Nov 04 23:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_341(8159350) done Nov 04 23:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_28(8158660) WEXITSTATUS 0 Nov 04 23:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_28(8158660) done Nov 04 23:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5412(8159393) WEXITSTATUS 1 Nov 04 23:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5412(8159393) done Nov 04 23:52:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1456(8137385) has been requeued to reclaim resources for JobId=8160535 Nov 04 23:52:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_342(8159351) WEXITSTATUS 0 Nov 04 23:52:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_342(8159351) done Nov 04 23:52:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_361(8159370) WEXITSTATUS 0 Nov 04 23:52:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_361(8159370) done Nov 04 23:52:06 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4078(8148345) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:52:06 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157711 had to be killed Nov 04 23:52:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1456(8137385) Nov 04 23:52:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8160535 NodeList=r801u06n03 #CPUs=4 Partition=pi_esi Nov 04 23:52:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4078(8148345) Nov 04 23:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_475(8159517) WEXITSTATUS 0 Nov 04 23:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_475(8159517) done Nov 04 23:52:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_295(8159279) WEXITSTATUS 0 Nov 04 23:52:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_295(8159279) done Nov 04 23:52:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2248(8152165) WEXITSTATUS 0 Nov 04 23:52:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2248(8152165) done Nov 04 23:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_420(8159447) WEXITSTATUS 0 Nov 04 23:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_420(8159447) done Nov 04 23:52:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_352(8159361) WEXITSTATUS 0 Nov 04 23:52:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_352(8159361) done Nov 04 23:52:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_2(8158961) WEXITSTATUS 0 Nov 04 23:52:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_2(8158961) done Nov 04 23:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5515(8160518) WEXITSTATUS 1 Nov 04 23:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5515(8160518) done Nov 04 23:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_467(8159509) WEXITSTATUS 0 Nov 04 23:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_467(8159509) done Nov 04 23:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_480(8159522) WEXITSTATUS 0 Nov 04 23:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_480(8159522) done Nov 04 23:52:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_340(8159349) WEXITSTATUS 0 Nov 04 23:52:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_340(8159349) done Nov 04 23:52:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_474(8159516) WEXITSTATUS 0 Nov 04 23:52:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_474(8159516) done Nov 04 23:52:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_481(8159523) WEXITSTATUS 0 Nov 04 23:52:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_481(8159523) done Nov 04 23:52:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_362(8159371) WEXITSTATUS 0 Nov 04 23:52:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_362(8159371) done Nov 04 23:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_10(8158905) WEXITSTATUS 0 Nov 04 23:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_10(8158905) done Nov 04 23:52:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:52:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_363(8159372) WEXITSTATUS 0 Nov 04 23:52:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_363(8159372) done Nov 04 23:52:41 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 23:52:41 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 04 23:52:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_373(8159400) WEXITSTATUS 0 Nov 04 23:52:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_373(8159400) done Nov 04 23:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_355(8159364) WEXITSTATUS 0 Nov 04 23:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_355(8159364) done Nov 04 23:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_472(8159514) WEXITSTATUS 0 Nov 04 23:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_472(8159514) done Nov 04 23:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_469(8159511) WEXITSTATUS 0 Nov 04 23:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_469(8159511) done Nov 04 23:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_479(8159521) WEXITSTATUS 0 Nov 04 23:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_479(8159521) done Nov 04 23:52:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_427(8159454) WEXITSTATUS 0 Nov 04 23:52:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_427(8159454) done Nov 04 23:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157497 WEXITSTATUS 0 Nov 04 23:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157497 done Nov 04 23:52:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_4(8158963) WEXITSTATUS 0 Nov 04 23:52:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_4(8158963) done Nov 04 23:52:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5184(8158538) WEXITSTATUS 1 Nov 04 23:52:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5184(8158538) done Nov 04 23:52:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_316(8159300) WEXITSTATUS 0 Nov 04 23:52:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_316(8159300) done Nov 04 23:52:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_384(8159411) WEXITSTATUS 0 Nov 04 23:52:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_384(8159411) done Nov 04 23:53:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_476(8159518) WEXITSTATUS 0 Nov 04 23:53:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_476(8159518) done Nov 04 23:53:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1251(8135153) WEXITSTATUS 0 Nov 04 23:53:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1251(8135153) done Nov 04 23:53:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158774 Nov 04 23:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_343(8159352) WEXITSTATUS 0 Nov 04 23:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_343(8159352) done Nov 04 23:53:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5303(8158765) WEXITSTATUS 1 Nov 04 23:53:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5303(8158765) done Nov 04 23:53:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_441(8159483) WEXITSTATUS 0 Nov 04 23:53:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_441(8159483) done Nov 04 23:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_389(8159416) WEXITSTATUS 0 Nov 04 23:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_389(8159416) done Nov 04 23:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_482(8159524) WEXITSTATUS 0 Nov 04 23:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_482(8159524) done Nov 04 23:53:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5497(8160334) WEXITSTATUS 1 Nov 04 23:53:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5497(8160334) done Nov 04 23:53:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_470(8159512) WEXITSTATUS 0 Nov 04 23:53:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_470(8159512) done Nov 04 23:53:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3418(8135246) WEXITSTATUS 1 Nov 04 23:53:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3418(8135246) done Nov 04 23:53:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_305(8159289) WEXITSTATUS 0 Nov 04 23:53:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_305(8159289) done Nov 04 23:53:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_382(8159409) WEXITSTATUS 0 Nov 04 23:53:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_382(8159409) done Nov 04 23:53:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_432(8159474) WEXITSTATUS 0 Nov 04 23:53:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_432(8159474) done Nov 04 23:53:36 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_599(7572768) Nov 04 23:53:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_436(8159478) WEXITSTATUS 0 Nov 04 23:53:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_436(8159478) done Nov 04 23:53:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_338(8159347) WEXITSTATUS 0 Nov 04 23:53:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_338(8159347) done Nov 04 23:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5423(8159468) WEXITSTATUS 1 Nov 04 23:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5423(8159468) done Nov 04 23:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_489(8159531) WEXITSTATUS 0 Nov 04 23:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_489(8159531) done Nov 04 23:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_423(8159450) WEXITSTATUS 0 Nov 04 23:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_423(8159450) done Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157492 WEXITSTATUS 0 Nov 04 23:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157492 done Nov 04 23:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_466(8159508) WEXITSTATUS 0 Nov 04 23:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_466(8159508) done Nov 04 23:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_502(8159544) WEXITSTATUS 0 Nov 04 23:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_502(8159544) done Nov 04 23:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_371(8159398) WEXITSTATUS 0 Nov 04 23:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_371(8159398) done Nov 04 23:54:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_428(8159470) WEXITSTATUS 0 Nov 04 23:54:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_428(8159470) done Nov 04 23:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_6(8158965) WEXITSTATUS 0 Nov 04 23:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_6(8158965) done Nov 04 23:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_360(8159369) WEXITSTATUS 0 Nov 04 23:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_360(8159369) done Nov 04 23:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_485(8159527) WEXITSTATUS 0 Nov 04 23:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_485(8159527) done Nov 04 23:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_357(8159366) WEXITSTATUS 0 Nov 04 23:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_357(8159366) done Nov 04 23:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_429(8159471) WEXITSTATUS 0 Nov 04 23:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_429(8159471) done Nov 04 23:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_477(8159519) WEXITSTATUS 0 Nov 04 23:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_477(8159519) done Nov 04 23:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5401(8159339) WEXITSTATUS 1 Nov 04 23:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5401(8159339) done Nov 04 23:54:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_3(8158962) WEXITSTATUS 0 Nov 04 23:54:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_3(8158962) done Nov 04 23:54:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_559(8159620) WEXITSTATUS 0 Nov 04 23:54:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_559(8159620) done Nov 04 23:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_358(8159367) WEXITSTATUS 0 Nov 04 23:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_358(8159367) done Nov 04 23:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_378(8159405) WEXITSTATUS 0 Nov 04 23:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_378(8159405) done Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_441(8160881) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_442(8160882) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_443(8160883) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_444(8160884) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_445(8160885) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_446(8160886) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_447(8160887) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_448(8160888) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_449(8160889) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_450(8160890) in pi_ohern on r917u13n01 Nov 04 23:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_451(8160891) in pi_ohern on r917u13n02 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_421(8159448) WEXITSTATUS 0 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_421(8159448) done Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_452(8160892) in pi_ohern on r917u13n02 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_453(8160893) in pi_ohern on r917u13n02 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_454(8160894) in pi_ohern on r917u13n03 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_455(8160895) in pi_ohern on r917u13n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_456(8160896) in pi_ohern on r917u29n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_457(8160897) in pi_ohern on r917u29n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_458(8160898) in pi_ohern on r917u29n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_459(8160899) in pi_ohern on r917u29n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_460(8160900) in pi_ohern on r917u29n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_461(8160901) in pi_ohern on r917u29n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_462(8160902) in pi_ohern on r917u29n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_463(8160903) in pi_ohern on r917u29n02 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_464(8160904) in pi_ohern on r917u29n02 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_465(8160905) in pi_ohern on r917u29n02 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_466(8160906) in pi_ohern on r917u29n02 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_467(8160907) in pi_ohern on r917u29n02 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_468(8160908) in pi_ohern on r917u29n03 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_469(8160909) in pi_ohern on r917u29n03 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_470(8160910) in pi_ohern on r917u29n03 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_471(8160911) in pi_ohern on r917u29n03 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_472(8160912) in pi_ohern on r917u29n03 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_473(8160913) in pi_ohern on r917u29n03 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_474(8160914) in pi_ohern on r917u29n03 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_475(8160915) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_476(8160916) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_477(8160917) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_478(8160918) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_479(8160919) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_480(8160920) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_481(8160921) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_482(8160922) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_483(8160923) in pi_ohern on r917u29n04 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_484(8160924) in pi_ohern on r815u03n11 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_485(8160925) in pi_ohern on r815u03n11 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_486(8160926) in pi_ohern on r815u03n11 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_487(8160927) in pi_ohern on r815u03n11 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_488(8160928) in pi_ohern on r815u03n11 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_489(8160929) in pi_ohern on r807u24n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_490(8160930) in pi_ohern on r911u32n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_491(8160931) in pi_ohern on r913u32n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_492(8160932) in pi_ohern on r913u32n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_493(8160933) in pi_ohern on r913u35n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_494(8160934) in pi_ohern on r913u35n01 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159908 in day on r815u09n06 Nov 04 23:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159909 in day on r815u23n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_500(8160935) in day on r807u11n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_501(8160936) in day on r807u11n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_502(8160937) in day on r909u29n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_503(8160938) in day on r909u29n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_504(8160939) in day on r805u04n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_505(8160940) in day on r805u06n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_506(8160941) in day on r805u06n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_507(8160942) in day on r805u06n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_508(8160943) in day on r805u06n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_509(8160944) in day on r805u06n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_510(8160945) in day on r805u06n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_511(8160946) in day on r805u06n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_512(8160947) in day on r805u06n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_513(8160948) in day on r805u06n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_514(8160949) in day on r805u06n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_515(8160950) in day on r805u06n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_516(8160951) in day on r805u06n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_517(8160952) in day on r805u06n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_518(8160953) in day on r805u06n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_519(8160954) in day on r805u06n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_520(8160955) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_521(8160956) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_522(8160957) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_523(8160958) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_524(8160959) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_525(8160960) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_526(8160961) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_527(8160962) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_528(8160963) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_529(8160964) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_530(8160965) in day on r805u09n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_531(8160966) in day on r805u09n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_532(8160967) in day on r805u09n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_533(8160968) in day on r805u09n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_534(8160969) in day on r805u09n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_535(8160970) in day on r805u09n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_536(8160971) in day on r805u09n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_537(8160972) in day on r805u09n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_538(8160973) in day on r805u09n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_539(8160974) in day on r805u09n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_540(8160975) in day on r805u09n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_541(8160976) in day on r805u11n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_542(8160977) in day on r805u11n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_543(8160978) in day on r805u11n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_544(8160979) in day on r805u11n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_545(8160980) in day on r805u11n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_546(8160981) in day on r805u11n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_547(8160982) in day on r805u11n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_548(8160983) in day on r805u11n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_549(8160984) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_550(8160985) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_551(8160986) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_552(8160987) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_553(8160988) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_554(8160989) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_555(8160990) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_556(8160991) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_557(8160992) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_558(8160993) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_559(8160994) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_560(8160995) in day on r805u11n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_561(8160996) in day on r805u11n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_562(8160997) in day on r805u11n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_563(8160998) in day on r805u11n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_564(8160999) in day on r805u11n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_565(8161000) in day on r805u11n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_566(8161001) in day on r805u14n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_567(8161002) in day on r805u14n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_568(8161003) in day on r805u14n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_569(8161004) in day on r805u14n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_570(8161005) in day on r805u14n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_571(8161006) in day on r805u14n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_572(8161007) in day on r805u14n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_573(8161008) in day on r805u14n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_574(8161009) in day on r805u14n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_575(8161010) in day on r805u14n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_576(8161011) in day on r805u14n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_577(8161012) in day on r805u14n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_578(8161013) in day on r805u14n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_579(8161014) in day on r805u14n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_580(8161015) in day on r805u14n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_581(8161016) in day on r805u14n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_582(8161017) in day on r805u23n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_583(8161018) in day on r805u23n03 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_584(8161019) in day on r805u23n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_585(8161020) in day on r805u23n04 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_586(8161021) in day on r805u25n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_587(8161022) in day on r805u25n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_588(8161023) in day on r805u25n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_589(8161024) in day on r805u25n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_590(8161025) in day on r805u25n01 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_591(8161026) in day on r805u25n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_592(8161027) in day on r805u25n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_593(8161028) in day on r805u25n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_594(8161029) in day on r805u25n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_595(8161030) in day on r805u25n02 Nov 04 23:54:23 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:192 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 04 23:54:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_596(8161031) in day on r805u25n03 Nov 04 23:54:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_597(8161032) in day on r805u25n04 Nov 04 23:54:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_598(8161033) in day on r805u25n04 Nov 04 23:54:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_599(8161034) in day on r805u25n04 Nov 04 23:54:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_25(8161035) in day on r815u15n04 Nov 04 23:54:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_383(8159410) WEXITSTATUS 0 Nov 04 23:54:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_383(8159410) done Nov 04 23:54:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_498(8159540) WEXITSTATUS 0 Nov 04 23:54:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_498(8159540) done Nov 04 23:54:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_491(8159533) WEXITSTATUS 0 Nov 04 23:54:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_491(8159533) done Nov 04 23:54:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_468(8159510) WEXITSTATUS 0 Nov 04 23:54:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_468(8159510) done Nov 04 23:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_488(8159530) WEXITSTATUS 0 Nov 04 23:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_488(8159530) done Nov 04 23:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_442(8159484) WEXITSTATUS 0 Nov 04 23:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_442(8159484) done Nov 04 23:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_336(8159345) WEXITSTATUS 0 Nov 04 23:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_336(8159345) done Nov 04 23:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5141(8158488) WEXITSTATUS 1 Nov 04 23:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5141(8158488) done Nov 04 23:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_5(8158964) WEXITSTATUS 0 Nov 04 23:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_5(8158964) done Nov 04 23:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_430(8159472) WEXITSTATUS 0 Nov 04 23:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_430(8159472) done Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5546(8161036) in scavenge on r911u05n01 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5547(8161037) in scavenge on r815u03n09 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5548(8161038) in scavenge on r816u29n10 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5549(8161039) in scavenge on r816u29n10 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5550(8161040) in scavenge on r808u08n01 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5551(8161041) in scavenge on r816u15n03 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5552(8161042) in scavenge on r816u15n03 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5553(8161043) in scavenge on r816u15n03 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5554(8161044) in scavenge on r815u09n01 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5555(8161045) in scavenge on r815u09n06 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5556(8161046) in scavenge on r815u23n04 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5557(8161047) in scavenge on r815u23n04 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5558(8161048) in scavenge on r815u29n03 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5559(8161049) in scavenge on r805u34n01 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_497(8159539) WEXITSTATUS 0 Nov 04 23:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_497(8159539) done Nov 04 23:54:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_316(7572484) in scavenge on r816u29n05 Nov 04 23:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5541(8160876) WEXITSTATUS 1 Nov 04 23:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5541(8160876) done Nov 04 23:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161050 InitPrio=27520 usec=2979 Nov 04 23:54:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8138596 WEXITSTATUS 0 Nov 04 23:54:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8138596 done Nov 04 23:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_443(8159485) WEXITSTATUS 0 Nov 04 23:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_443(8159485) done Nov 04 23:54:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_495(8159537) WEXITSTATUS 0 Nov 04 23:54:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_495(8159537) done Nov 04 23:54:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161050 NodeList=r906u01n[01,03-04],r907u01n01,r907u09n[01,04] #CPUs=144 Partition=pi_fedorov Nov 04 23:54:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_334(8159343) WEXITSTATUS 0 Nov 04 23:54:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_334(8159343) done Nov 04 23:54:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_527(8159588) WEXITSTATUS 0 Nov 04 23:54:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_527(8159588) done Nov 04 23:54:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5403(8159384) WEXITSTATUS 1 Nov 04 23:54:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5403(8159384) done Nov 04 23:54:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_473(8159515) WEXITSTATUS 0 Nov 04 23:54:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_473(8159515) done Nov 04 23:54:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_388(8159415) WEXITSTATUS 0 Nov 04 23:54:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_388(8159415) done Nov 04 23:55:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_431(8159473) WEXITSTATUS 0 Nov 04 23:55:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_431(8159473) done Nov 04 23:55:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_448(8159490) WEXITSTATUS 0 Nov 04 23:55:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_448(8159490) done Nov 04 23:55:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5557(8161047) WEXITSTATUS 1 Nov 04 23:55:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5557(8161047) done Nov 04 23:55:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157496 WEXITSTATUS 0 Nov 04 23:55:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157496 done Nov 04 23:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_438(8159480) WEXITSTATUS 0 Nov 04 23:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_438(8159480) done Nov 04 23:55:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_446(8159488) WEXITSTATUS 0 Nov 04 23:55:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_446(8159488) done Nov 04 23:55:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_447(8159489) WEXITSTATUS 0 Nov 04 23:55:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_447(8159489) done Nov 04 23:55:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_493(8159535) WEXITSTATUS 0 Nov 04 23:55:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_493(8159535) done Nov 04 23:55:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5390(8159328) WEXITSTATUS 1 Nov 04 23:55:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5390(8159328) done Nov 04 23:55:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157495 WEXITSTATUS 0 Nov 04 23:55:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157495 done Nov 04 23:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_386(8159413) WEXITSTATUS 0 Nov 04 23:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_386(8159413) done Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3073(8130388) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3149(8131207) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3990(8146254) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4122(8149428) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4520(8155921) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4768(8157450) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4970(8158232) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4973(8158235) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4992(8158271) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5227(8158630) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5312(8158840) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5349(8159083) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159028_3(8159095) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159028_4(8159096) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159028_5(8159097) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159028_9(8159101) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159028_10(8159028) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159102 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159118_1(8159182) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159118_5(8159186) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159118_6(8159187) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159118_7(8159188) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159196 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159198 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159205 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159207 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159208 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159209 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159210 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159211 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159212 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159213 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159214 has been requeued to reclaim resources for JobId=Invalid Nov 04 23:55:39 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3149(8131207) Nov 04 23:55:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3073(8130388) Nov 04 23:55:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4122(8149428) Nov 04 23:55:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4768(8157450) Nov 04 23:55:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159028_10(8159028) Nov 04 23:55:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159213 Nov 04 23:55:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159207 Nov 04 23:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_385(8159412) WEXITSTATUS 0 Nov 04 23:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_385(8159412) done Nov 04 23:55:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159118_5(8159186) Nov 04 23:55:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4520(8155921) Nov 04 23:55:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159118_1(8159182) Nov 04 23:55:46 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159196 Nov 04 23:55:46 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159102 Nov 04 23:55:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159208 Nov 04 23:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_478(8159520) WEXITSTATUS 0 Nov 04 23:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_478(8159520) done Nov 04 23:55:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159210 Nov 04 23:55:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3990(8146254) Nov 04 23:55:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159198 Nov 04 23:55:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159209 Nov 04 23:55:49 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159028_5(8159097) Nov 04 23:55:49 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4970(8158232) Nov 04 23:55:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_7(8158966) WEXITSTATUS 0 Nov 04 23:55:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_7(8158966) done Nov 04 23:55:49 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5312(8158840) Nov 04 23:55:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159118_6(8159187) Nov 04 23:55:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_374(8159401) WEXITSTATUS 0 Nov 04 23:55:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_374(8159401) done Nov 04 23:55:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159118_7(8159188) Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5349(8159083) Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:55:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159205 Nov 04 23:55:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159028_4(8159096) Nov 04 23:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_351(8159360) WEXITSTATUS 0 Nov 04 23:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_351(8159360) done Nov 04 23:55:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159212 Nov 04 23:55:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4973(8158235) Nov 04 23:55:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5227(8158630) Nov 04 23:55:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159214 Nov 04 23:55:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159028_9(8159101) Nov 04 23:55:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159211 Nov 04 23:55:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159028_3(8159095) Nov 04 23:55:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_359(8159368) WEXITSTATUS 0 Nov 04 23:55:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_359(8159368) done Nov 04 23:55:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4992(8158271) Nov 04 23:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_528(8159589) WEXITSTATUS 0 Nov 04 23:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_528(8159589) done Nov 04 23:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4886(8157997) WEXITSTATUS 1 Nov 04 23:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4886(8157997) done Nov 04 23:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_9(8158968) WEXITSTATUS 0 Nov 04 23:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_9(8158968) done Nov 04 23:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_8(8158967) WEXITSTATUS 0 Nov 04 23:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158905_8(8158967) done Nov 04 23:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_381(8159408) WEXITSTATUS 0 Nov 04 23:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_381(8159408) done Nov 04 23:56:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_516(8159577) WEXITSTATUS 0 Nov 04 23:56:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_516(8159577) done Nov 04 23:56:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_370(8159397) WEXITSTATUS 0 Nov 04 23:56:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_370(8159397) done Nov 04 23:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5467(8159890) WEXITSTATUS 1 Nov 04 23:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5467(8159890) done Nov 04 23:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_492(8159534) WEXITSTATUS 0 Nov 04 23:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_492(8159534) done Nov 04 23:56:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_617(7572786) Nov 04 23:56:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_440(8159482) WEXITSTATUS 0 Nov 04 23:56:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_440(8159482) done Nov 04 23:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5457(8159773) WEXITSTATUS 1 Nov 04 23:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5457(8159773) done Nov 04 23:56:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_558(8159619) WEXITSTATUS 0 Nov 04 23:56:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_558(8159619) done Nov 04 23:56:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_568(8159629) WEXITSTATUS 0 Nov 04 23:56:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_568(8159629) done Nov 04 23:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_435(8159477) WEXITSTATUS 0 Nov 04 23:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_435(8159477) done Nov 04 23:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_335(8159344) WEXITSTATUS 0 Nov 04 23:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_335(8159344) done Nov 04 23:56:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_506(8159567) WEXITSTATUS 0 Nov 04 23:56:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_506(8159567) done Nov 04 23:56:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161051 InitPrio=34888 usec=3314 Nov 04 23:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_494(8159536) WEXITSTATUS 0 Nov 04 23:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_494(8159536) done Nov 04 23:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_437(8159479) WEXITSTATUS 0 Nov 04 23:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_437(8159479) done Nov 04 23:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_496(8159538) WEXITSTATUS 0 Nov 04 23:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_496(8159538) done Nov 04 23:56:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_390(8159417) WEXITSTATUS 0 Nov 04 23:56:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_390(8159417) done Nov 04 23:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_501(8159543) WEXITSTATUS 0 Nov 04 23:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_501(8159543) done Nov 04 23:56:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_376(8159403) WEXITSTATUS 0 Nov 04 23:56:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_376(8159403) done Nov 04 23:56:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_490(8159532) WEXITSTATUS 0 Nov 04 23:56:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_490(8159532) done Nov 04 23:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5183(8158537) WEXITSTATUS 1 Nov 04 23:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5183(8158537) done Nov 04 23:57:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_525(8159586) WEXITSTATUS 0 Nov 04 23:57:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_525(8159586) done Nov 04 23:57:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_391(8159418) WEXITSTATUS 0 Nov 04 23:57:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_391(8159418) done Nov 04 23:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157499 WEXITSTATUS 0 Nov 04 23:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157499 done Nov 04 23:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5177(8158531) WEXITSTATUS 1 Nov 04 23:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5177(8158531) done Nov 04 23:57:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1270(8135208) WEXITSTATUS 0 Nov 04 23:57:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1270(8135208) done Nov 04 23:57:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4200(8152038) has been requeued to reclaim resources for JobId=Invalid Nov 04 23:57:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4200(8152038) Nov 04 23:57:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_380(8159407) WEXITSTATUS 0 Nov 04 23:57:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_380(8159407) done Nov 04 23:57:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161051 NodeList=r801u06n01 #CPUs=4 Partition=education Nov 04 23:57:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_372(8159399) WEXITSTATUS 0 Nov 04 23:57:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_372(8159399) done Nov 04 23:57:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3609(8160693) WEXITSTATUS 0 Nov 04 23:57:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3609(8160693) done Nov 04 23:57:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3574(8159161) WEXITSTATUS 0 Nov 04 23:57:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3574(8159161) done Nov 04 23:57:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_509(8159570) WEXITSTATUS 0 Nov 04 23:57:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_509(8159570) done Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_495(8161052) in pi_ohern on r917u13n01 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_496(8161053) in pi_ohern on r917u13n01 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_497(8161054) in pi_ohern on r917u13n02 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_498(8161055) in pi_ohern on r917u13n02 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_499(8161056) in pi_ohern on r917u13n02 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_500(8161057) in pi_ohern on r917u13n02 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_501(8161058) in pi_ohern on r917u13n03 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_502(8161059) in pi_ohern on r917u13n03 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_503(8161060) in pi_ohern on r917u13n03 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_504(8161061) in pi_ohern on r917u13n03 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_505(8161062) in pi_ohern on r917u13n04 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_506(8161063) in pi_ohern on r917u13n04 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_507(8161064) in pi_ohern on r917u13n04 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_508(8161065) in pi_ohern on r917u13n04 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_509(8161066) in pi_ohern on r917u13n04 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_510(8161067) in pi_ohern on r917u13n04 Nov 04 23:57:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_511(8161068) in pi_ohern on r917u13n04 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_451(8159493) WEXITSTATUS 0 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_451(8159493) done Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_512(8161069) in pi_ohern on r917u13n04 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_513(8161070) in pi_ohern on r917u29n02 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_514(8161071) in pi_ohern on r917u29n02 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_515(8161072) in pi_ohern on r917u29n02 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_516(8161073) in pi_ohern on r917u29n02 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_517(8161074) in pi_ohern on r917u29n02 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_518(8161075) in pi_ohern on r917u29n03 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_519(8161076) in pi_ohern on r917u29n03 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_520(8161077) in pi_ohern on r917u29n03 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_521(8161078) in pi_ohern on r917u29n03 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_522(8161079) in pi_ohern on r917u29n04 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_523(8161080) in pi_ohern on r815u03n11 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_524(8161081) in pi_ohern on r815u03n11 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_525(8161082) in pi_ohern on r815u03n11 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_526(8161083) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_527(8161084) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_528(8161085) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_529(8161086) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_530(8161087) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_531(8161088) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_532(8161089) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_533(8161090) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_534(8161091) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_535(8161092) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_536(8161093) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_537(8161094) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_538(8161095) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_539(8161096) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_540(8161097) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_541(8161098) in pi_ohern on r815u03n12 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_542(8161099) in pi_ohern on r807u22n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_543(8161100) in pi_ohern on r807u24n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_544(8161101) in pi_ohern on r903u12n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_545(8161102) in pi_ohern on r903u12n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_546(8161103) in pi_ohern on r903u12n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_547(8161104) in pi_ohern on r903u12n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_548(8161105) in pi_ohern on r903u26n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_549(8161106) in pi_ohern on r903u26n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_550(8161107) in pi_ohern on r903u26n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_551(8161108) in pi_ohern on r913u32n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_552(8161109) in pi_ohern on r913u32n01 Nov 04 23:57:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_553(8161110) in pi_ohern on r913u35n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_600(8161111) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_601(8161112) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_602(8161113) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_603(8161114) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_604(8161115) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_605(8161116) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_606(8161117) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_607(8161118) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_608(8161119) in day on r912u28n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_609(8161120) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_610(8161121) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_611(8161122) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_612(8161123) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_613(8161124) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_614(8161125) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_615(8161126) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_616(8161127) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_617(8161128) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_618(8161129) in day on r904u10n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_619(8161130) in day on r909u04n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_620(8161131) in day on r909u04n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_621(8161132) in day on r909u04n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_622(8161133) in day on r909u29n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_623(8161134) in day on r909u29n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_624(8161135) in day on r909u29n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_625(8161136) in day on r805u25n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_626(8161137) in day on r805u25n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_627(8161138) in day on r805u27n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_628(8161139) in day on r805u27n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_629(8161140) in day on r805u27n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_630(8161141) in day on r805u27n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_631(8161142) in day on r805u27n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_632(8161143) in day on r805u27n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_633(8161144) in day on r918u01n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_634(8161145) in day on r918u01n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_635(8161146) in day on r918u01n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_636(8161147) in day on r918u01n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_637(8161148) in day on r918u01n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_638(8161149) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_639(8161150) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_640(8161151) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_641(8161152) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_642(8161153) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_643(8161154) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_644(8161155) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_645(8161156) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_646(8161157) in day on r918u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_647(8161158) in day on r918u05n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_648(8161159) in day on r918u05n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_649(8161160) in day on r918u05n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_650(8161161) in day on r918u05n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_651(8161162) in day on r918u05n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_652(8161163) in day on r918u05n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_653(8161164) in day on r918u05n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_654(8161165) in day on r918u05n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_655(8161166) in day on r918u09n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_656(8161167) in day on r918u09n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_657(8161168) in day on r918u09n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_658(8161169) in day on r918u09n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_659(8161170) in day on r918u32n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_660(8161171) in day on r918u32n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_661(8161172) in day on r918u32n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_662(8161173) in day on r918u32n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_663(8161174) in day on r918u32n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_664(8161175) in day on r918u32n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_665(8161176) in day on r918u32n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_666(8161177) in day on r918u32n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_667(8161178) in day on r918u32n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_668(8161179) in day on r918u36n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_669(8161180) in day on r918u36n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_670(8161181) in day on r918u36n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_671(8161182) in day on r919u01n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_672(8161183) in day on r919u01n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_673(8161184) in day on r919u01n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_674(8161185) in day on r919u01n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_675(8161186) in day on r919u01n01 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_676(8161187) in day on r919u01n02 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_677(8161188) in day on r919u01n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_678(8161189) in day on r919u01n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_679(8161190) in day on r919u01n03 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_680(8161191) in day on r919u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_681(8161192) in day on r919u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_682(8161193) in day on r919u01n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_683(8161194) in day on r919u05n04 Nov 04 23:57:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_684(8161195) in day on r919u05n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_685(8161196) in day on r919u09n01 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_686(8161197) in day on r919u09n01 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_687(8161198) in day on r919u09n01 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_688(8161199) in day on r919u09n02 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_689(8161200) in day on r919u09n02 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_387(8159414) WEXITSTATUS 0 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_387(8159414) done Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_690(8161201) in day on r919u09n03 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_691(8161202) in day on r919u09n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_692(8161203) in day on r919u09n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_693(8161204) in day on r919u09n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_694(8161205) in day on r919u09n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_695(8161206) in day on r919u09n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_696(8161207) in day on r919u09n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_697(8161208) in day on r919u09n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_698(8161209) in day on r919u09n04 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_699(8161210) in day on r919u32n01 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3612(8161211) in gpu on r808u17n01 Nov 04 23:57:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3613(8161212) in gpu on r4513u26n01 Nov 04 23:57:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1866(8161213) in day on r815u29n01 Nov 04 23:57:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r909u07n01 Nov 04 23:57:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_356(8159365) WEXITSTATUS 0 Nov 04 23:57:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_356(8159365) done Nov 04 23:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_510(8159571) WEXITSTATUS 0 Nov 04 23:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_510(8159571) done Nov 04 23:57:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_560(8159621) WEXITSTATUS 0 Nov 04 23:57:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_560(8159621) done Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5560(8161214) in scavenge on r906u01n02 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5561(8161215) in scavenge on r906u01n02 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5562(8161216) in scavenge on r907u01n02 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5563(8161217) in scavenge on r907u01n02 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5564(8161218) in scavenge on r907u01n04 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5565(8161219) in scavenge on r907u01n04 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5566(8161220) in scavenge on r907u05n02 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5567(8161221) in scavenge on r907u05n02 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5568(8161222) in scavenge on r907u05n04 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5569(8161223) in scavenge on r907u05n04 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5570(8161224) in scavenge on r907u09n02 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5571(8161225) in scavenge on r907u09n02 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5572(8161226) in scavenge on r802u02n03 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5573(8161227) in scavenge on r802u04n03 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5574(8161228) in scavenge on r815u09n12 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5575(8161229) in scavenge on r815u23n04 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5576(8161230) in scavenge on r816u35n05 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5577(8161231) in scavenge on r816u35n05 Nov 04 23:57:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5578(8161232) in scavenge on r816u35n06 Nov 04 23:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_507(8159568) WEXITSTATUS 0 Nov 04 23:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_507(8159568) done Nov 04 23:57:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_26(8161233) in scavenge on r816u23n09 Nov 04 23:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3598(8159651) WEXITSTATUS 0 Nov 04 23:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3598(8159651) done Nov 04 23:57:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_503(8159564) WEXITSTATUS 0 Nov 04 23:57:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_503(8159564) done Nov 04 23:57:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_562(8159623) WEXITSTATUS 0 Nov 04 23:57:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_562(8159623) done Nov 04 23:57:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_453(8159495) WEXITSTATUS 0 Nov 04 23:57:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_453(8159495) done Nov 04 23:57:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161234 InitPrio=61739 usec=9220 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_433(8159475) WEXITSTATUS 0 Nov 04 23:57:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_433(8159475) done Nov 04 23:58:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_512(8159573) WEXITSTATUS 0 Nov 04 23:58:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_512(8159573) done Nov 04 23:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_375(8159402) WEXITSTATUS 0 Nov 04 23:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_375(8159402) done Nov 04 23:58:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_573(8159634) WEXITSTATUS 0 Nov 04 23:58:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_573(8159634) done Nov 04 23:58:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161235 InitPrio=40248 usec=8018 Nov 04 23:58:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_379(8159406) WEXITSTATUS 0 Nov 04 23:58:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_379(8159406) done Nov 04 23:58:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4824(8157803) WEXITSTATUS 1 Nov 04 23:58:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4824(8157803) done Nov 04 23:58:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_487(8159529) WEXITSTATUS 0 Nov 04 23:58:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_487(8159529) done Nov 04 23:58:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_499(8159541) WEXITSTATUS 0 Nov 04 23:58:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_499(8159541) done Nov 04 23:58:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_444(8159486) WEXITSTATUS 0 Nov 04 23:58:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_444(8159486) done Nov 04 23:58:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5484(8160133) WEXITSTATUS 1 Nov 04 23:58:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5484(8160133) done Nov 04 23:58:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8160141 uid 20398 Nov 04 23:58:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_486(8159528) WEXITSTATUS 0 Nov 04 23:58:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_486(8159528) done Nov 04 23:58:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161234 NodeList=r918u01n01 #CPUs=1 Partition=day Nov 04 23:58:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161235 NodeList=r914u01n02 #CPUs=8 Partition=day Nov 04 23:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157504 WEXITSTATUS 0 Nov 04 23:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157504 done Nov 04 23:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_459(8159501) WEXITSTATUS 0 Nov 04 23:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_459(8159501) done Nov 04 23:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157505 WEXITSTATUS 0 Nov 04 23:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157505 done Nov 04 23:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161234 WEXITSTATUS 127 Nov 04 23:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161234 done Nov 04 23:58:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_445(8159487) WEXITSTATUS 0 Nov 04 23:58:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_445(8159487) done Nov 04 23:58:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8161235 uid 20398 Nov 04 23:58:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161236 InitPrio=40248 usec=5649 Nov 04 23:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_572(8159633) WEXITSTATUS 0 Nov 04 23:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_572(8159633) done Nov 04 23:58:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4838(8157846) WEXITSTATUS 1 Nov 04 23:58:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4838(8157846) done Nov 04 23:58:39 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 04 23:58:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157506 WEXITSTATUS 0 Nov 04 23:58:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157506 done Nov 04 23:58:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_563(8159624) WEXITSTATUS 0 Nov 04 23:58:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_563(8159624) done Nov 04 23:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5271(8158705) WEXITSTATUS 1 Nov 04 23:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5271(8158705) done Nov 04 23:58:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_504(8159565) WEXITSTATUS 0 Nov 04 23:58:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_504(8159565) done Nov 04 23:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_569(8159630) WEXITSTATUS 0 Nov 04 23:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_569(8159630) done Nov 04 23:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2213(8152005) WEXITSTATUS 0 Nov 04 23:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2213(8152005) done Nov 04 23:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_450(8159492) WEXITSTATUS 0 Nov 04 23:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_450(8159492) done Nov 04 23:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5269(8158703) WEXITSTATUS 1 Nov 04 23:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5269(8158703) done Nov 04 23:58:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_392(8159419) WEXITSTATUS 0 Nov 04 23:58:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_392(8159419) done Nov 04 23:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156077 WEXITSTATUS 0 Nov 04 23:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156077 done Nov 04 23:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_461(8159503) WEXITSTATUS 0 Nov 04 23:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_461(8159503) done Nov 04 23:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_455(8159497) WEXITSTATUS 0 Nov 04 23:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_455(8159497) done Nov 04 23:59:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_566(8159627) WEXITSTATUS 0 Nov 04 23:59:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_566(8159627) done Nov 04 23:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_500(8159542) WEXITSTATUS 0 Nov 04 23:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_500(8159542) done Nov 04 23:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_16(8158590) WEXITSTATUS 0 Nov 04 23:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_16(8158590) done Nov 04 23:59:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_464(8159506) WEXITSTATUS 0 Nov 04 23:59:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_464(8159506) done Nov 04 23:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_439(8159481) WEXITSTATUS 0 Nov 04 23:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_439(8159481) done Nov 04 23:59:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_570(8159631) WEXITSTATUS 0 Nov 04 23:59:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_570(8159631) done Nov 04 23:59:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_508(8159569) WEXITSTATUS 0 Nov 04 23:59:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_508(8159569) done Nov 04 23:59:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_554(8159615) WEXITSTATUS 0 Nov 04 23:59:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_554(8159615) done Nov 04 23:59:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161236 NodeList=r805u06n02 #CPUs=8 Partition=day Nov 04 23:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_434(8159476) WEXITSTATUS 0 Nov 04 23:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_434(8159476) done Nov 04 23:59:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_518(8159579) WEXITSTATUS 0 Nov 04 23:59:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_518(8159579) done Nov 04 23:59:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_571(8159632) WEXITSTATUS 0 Nov 04 23:59:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_571(8159632) done Nov 04 23:59:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5337(8158956) WEXITSTATUS 1 Nov 04 23:59:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5337(8158956) done Nov 04 23:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3613(8161212) WEXITSTATUS 0 Nov 04 23:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3613(8161212) done Nov 04 23:59:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_463(8159505) WEXITSTATUS 0 Nov 04 23:59:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_463(8159505) done Nov 04 23:59:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4985(8158258) WEXITSTATUS 1 Nov 04 23:59:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4985(8158258) done Nov 04 23:59:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_465(8159507) WEXITSTATUS 0 Nov 04 23:59:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_465(8159507) done Nov 04 23:59:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_565(8159626) WEXITSTATUS 0 Nov 04 23:59:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_565(8159626) done Nov 04 23:59:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5249(8158673) WEXITSTATUS 1 Nov 04 23:59:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5249(8158673) done Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 04 23:59:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 04 23:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_519(8159580) WEXITSTATUS 0 Nov 04 23:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_519(8159580) done Nov 04 23:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_520(8159581) WEXITSTATUS 0 Nov 04 23:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_520(8159581) done Nov 04 23:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_511(8159572) WEXITSTATUS 0 Nov 04 23:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_511(8159572) done Nov 04 23:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3611(8160861) WEXITSTATUS 0 Nov 04 23:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3611(8160861) done Nov 04 23:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3612(8161211) WEXITSTATUS 0 Nov 04 23:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3612(8161211) done Nov 05 00:00:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_523(8159584) WEXITSTATUS 0 Nov 05 00:00:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_523(8159584) done Nov 05 00:00:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_530(8159591) WEXITSTATUS 0 Nov 05 00:00:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_530(8159591) done Nov 05 00:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 00:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 00:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 00:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 00:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 00:00:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4539(8155944) WEXITSTATUS 1 Nov 05 00:00:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4539(8155944) done Nov 05 00:00:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2168(8151960) WEXITSTATUS 0 Nov 05 00:00:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2168(8151960) done Nov 05 00:00:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_535(8159596) WEXITSTATUS 0 Nov 05 00:00:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_535(8159596) done Nov 05 00:00:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_524(8159585) WEXITSTATUS 0 Nov 05 00:00:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_524(8159585) done Nov 05 00:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_522(8159583) WEXITSTATUS 0 Nov 05 00:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_522(8159583) done Nov 05 00:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5058(8158363) WEXITSTATUS 1 Nov 05 00:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5058(8158363) done Nov 05 00:00:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1153(8132731) WEXITSTATUS 0 Nov 05 00:00:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1153(8132731) done Nov 05 00:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_457(8159499) WEXITSTATUS 0 Nov 05 00:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_457(8159499) done Nov 05 00:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_561(8159622) WEXITSTATUS 0 Nov 05 00:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_561(8159622) done Nov 05 00:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_454(8159496) WEXITSTATUS 0 Nov 05 00:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_454(8159496) done Nov 05 00:00:17 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:20 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_554(8161237) in pi_ohern on r917u13n01 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_555(8161238) in pi_ohern on r917u13n01 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_556(8161239) in pi_ohern on r917u13n01 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_557(8161240) in pi_ohern on r917u13n01 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_558(8161241) in pi_ohern on r917u13n01 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_559(8161242) in pi_ohern on r917u13n02 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_560(8161243) in pi_ohern on r917u13n02 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_561(8161244) in pi_ohern on r917u13n02 Nov 05 00:00:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_562(8161245) in pi_ohern on r917u13n02 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_563(8161246) in pi_ohern on r917u13n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_564(8161247) in pi_ohern on r917u13n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_565(8161248) in pi_ohern on r917u13n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_566(8161249) in pi_ohern on r917u13n04 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_567(8161250) in pi_ohern on r917u13n04 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_568(8161251) in pi_ohern on r917u13n04 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_569(8161252) in pi_ohern on r917u13n04 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_570(8161253) in pi_ohern on r917u13n04 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_571(8161254) in pi_ohern on r917u13n04 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_572(8161255) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_573(8161256) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_574(8161257) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_575(8161258) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_576(8161259) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_577(8161260) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_578(8161261) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_579(8161262) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_580(8161263) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_581(8161264) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_582(8161265) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_583(8161266) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_584(8161267) in pi_ohern on r917u29n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_585(8161268) in pi_ohern on r917u29n02 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_586(8161269) in pi_ohern on r917u29n02 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_587(8161270) in pi_ohern on r917u29n02 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_588(8161271) in pi_ohern on r917u29n02 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_589(8161272) in pi_ohern on r917u29n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_590(8161273) in pi_ohern on r917u29n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_591(8161274) in pi_ohern on r917u29n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_592(8161275) in pi_ohern on r917u29n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_593(8161276) in pi_ohern on r917u29n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_594(8161277) in pi_ohern on r917u29n03 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_595(8161278) in pi_ohern on r917u29n04 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_596(8161279) in pi_ohern on r917u29n04 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_597(8161280) in pi_ohern on r807u24n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_598(8161281) in pi_ohern on r807u24n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_599(8161282) in pi_ohern on r807u24n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_600(8161283) in pi_ohern on r807u24n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_601(8161284) in pi_ohern on r807u24n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_602(8161285) in pi_ohern on r911u32n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_603(8161286) in pi_ohern on r911u32n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_604(8161287) in pi_ohern on r913u35n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_605(8161288) in pi_ohern on r913u35n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_606(8161289) in pi_ohern on r903u12n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_607(8161290) in pi_ohern on r903u12n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_608(8161291) in pi_ohern on r903u26n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_609(8161292) in pi_ohern on r903u26n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_610(8161293) in pi_ohern on r903u26n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_611(8161294) in pi_ohern on r903u26n01 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r805u06n02 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r805u06n02 Nov 05 00:00:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159902 in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_700(8161295) in day on r805u06n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_701(8161296) in day on r805u06n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_702(8161297) in day on r805u06n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_703(8161298) in day on r805u06n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_704(8161299) in day on r805u06n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_705(8161300) in day on r805u06n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_706(8161301) in day on r805u06n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_707(8161302) in day on r805u06n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_708(8161303) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_709(8161304) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_710(8161305) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_711(8161306) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_712(8161307) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_713(8161308) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_714(8161309) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_715(8161310) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_716(8161311) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_717(8161312) in day on r914u01n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_718(8161313) in day on r904u07n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_719(8161314) in day on r904u07n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_720(8161315) in day on r906u41n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_721(8161316) in day on r912u28n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_722(8161317) in day on r912u28n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_723(8161318) in day on r912u28n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_724(8161319) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_725(8161320) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_726(8161321) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_727(8161322) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_728(8161323) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_729(8161324) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_730(8161325) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_731(8161326) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_732(8161327) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_733(8161328) in day on r912u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_734(8161329) in day on r919u09n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_735(8161330) in day on r919u09n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_736(8161331) in day on r919u09n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_737(8161332) in day on r919u32n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_738(8161333) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_739(8161334) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_740(8161335) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_741(8161336) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_742(8161337) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_743(8161338) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_744(8161339) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_745(8161340) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_746(8161341) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_747(8161342) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_748(8161343) in day on r919u32n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_749(8161344) in day on r919u32n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_750(8161345) in day on r919u32n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_751(8161346) in day on r919u32n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_752(8161347) in day on r919u32n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_753(8161348) in day on r919u32n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_754(8161349) in day on r919u32n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_755(8161350) in day on r919u32n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_756(8161351) in day on r919u32n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_757(8161352) in day on r919u32n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_758(8161353) in day on r919u36n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_759(8161354) in day on r919u36n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_760(8161355) in day on r919u36n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_761(8161356) in day on r919u36n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_762(8161357) in day on r919u36n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_763(8161358) in day on r919u36n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_764(8161359) in day on r919u36n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_765(8161360) in day on r919u36n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_766(8161361) in day on r919u36n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_767(8161362) in day on r919u36n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_768(8161363) in day on r919u36n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_769(8161364) in day on r919u36n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_770(8161365) in day on r919u36n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_771(8161366) in day on r919u36n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_772(8161367) in day on r919u36n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_773(8161368) in day on r919u36n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_774(8161369) in day on r815u23n01 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_775(8161370) in day on r815u23n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_776(8161371) in day on r815u23n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_777(8161372) in day on r815u23n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_778(8161373) in day on r815u23n02 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_779(8161374) in day on r815u23n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_780(8161375) in day on r815u23n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_781(8161376) in day on r815u23n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_782(8161377) in day on r815u23n03 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_783(8161378) in day on r815u23n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_784(8161379) in day on r815u23n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_785(8161380) in day on r815u23n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_786(8161381) in day on r815u23n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_787(8161382) in day on r815u23n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_788(8161383) in day on r815u23n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_789(8161384) in day on r815u23n04 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_790(8161385) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_791(8161386) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_792(8161387) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_793(8161388) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_794(8161389) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_795(8161390) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_796(8161391) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_797(8161392) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_798(8161393) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_799(8161394) in day on r815u23n05 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:308 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_607(7572776) WEXITSTATUS 0 Nov 05 00:00:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_607(7572776) done Nov 05 00:00:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3614(8161395) in gpu on r903u40n01 Nov 05 00:00:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3615(8161396) in gpu on r808u14n01 Nov 05 00:00:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3616(8161397) in gpu on r808u17n01 Nov 05 00:00:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3617(8161398) in gpu on r4513u26n01 Nov 05 00:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_113(7572280) WEXITSTATUS 0 Nov 05 00:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_113(7572280) done Nov 05 00:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5282(8158730) WEXITSTATUS 1 Nov 05 00:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5282(8158730) done Nov 05 00:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_703(7572872) WEXITSTATUS 0 Nov 05 00:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_703(7572872) done Nov 05 00:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_223(7572391) WEXITSTATUS 0 Nov 05 00:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_223(7572391) done Nov 05 00:00:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_514(8159575) WEXITSTATUS 0 Nov 05 00:00:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_514(8159575) done Nov 05 00:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5534(8160869) WEXITSTATUS 1 Nov 05 00:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5534(8160869) done Nov 05 00:00:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_377(8159404) WEXITSTATUS 0 Nov 05 00:00:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_377(8159404) done Nov 05 00:00:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_462(8159504) WEXITSTATUS 0 Nov 05 00:00:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_462(8159504) done Nov 05 00:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_505(8159566) WEXITSTATUS 0 Nov 05 00:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_505(8159566) done Nov 05 00:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_517(8159578) WEXITSTATUS 0 Nov 05 00:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_517(8159578) done Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159213 in scavenge on r906u01n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159207 in scavenge on r906u01n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159102 in scavenge on r906u01n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159196 in scavenge on r906u01n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159208 in scavenge on r908u28n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159210 in scavenge on r908u28n03 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159198 in scavenge on r908u37n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159209 in scavenge on r908u37n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159205 in scavenge on r908u37n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159212 in scavenge on r908u37n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159214 in scavenge on r908u37n02 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159211 in scavenge on r908u39n01 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5579(8161399) in scavenge on r903u07n01 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5580(8161400) in scavenge on r912u05n04 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5581(8161401) in scavenge on r917u10n01 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5582(8161402) in scavenge on r802u02n03 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5583(8161403) in scavenge on r802u06n01 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5584(8161404) in scavenge on r816u29n07 Nov 05 00:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5585(8161405) in scavenge on r816u09n03 Nov 05 00:00:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5586(8161406) in scavenge on r816u09n03 Nov 05 00:00:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5587(8161407) in scavenge on r816u35n04 Nov 05 00:00:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_324(7572492) in scavenge on r816u29n07 Nov 05 00:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_552(8159613) WEXITSTATUS 0 Nov 05 00:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_552(8159613) done Nov 05 00:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_449(8159491) WEXITSTATUS 0 Nov 05 00:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_449(8159491) done Nov 05 00:00:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_547(8159608) WEXITSTATUS 0 Nov 05 00:00:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_547(8159608) done Nov 05 00:00:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_544(8159605) WEXITSTATUS 0 Nov 05 00:00:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_544(8159605) done Nov 05 00:00:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_529(8159590) WEXITSTATUS 0 Nov 05 00:00:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_529(8159590) done Nov 05 00:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1867(8161408) in scavenge on r806u28n01 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_10(8159028) in scavenge on r908u39n03 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_5(8159186) in scavenge on r908u33n02 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_1(8159182) in scavenge on r908u33n03 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_5(8159097) in scavenge on r908u33n04 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_6(8159187) in scavenge on r801u30n01 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159118_7(8159188) in scavenge on r801u30n01 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_4(8159096) in scavenge on r801u30n01 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_9(8159101) in scavenge on r801u30n02 Nov 05 00:00:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159028_3(8159095) in scavenge on r801u30n02 Nov 05 00:00:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 00:00:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 00:00:58 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 00:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_18(8158592) WEXITSTATUS 0 Nov 05 00:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_18(8158592) done Nov 05 00:01:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_537(8159598) WEXITSTATUS 0 Nov 05 00:01:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_537(8159598) done Nov 05 00:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_538(8159599) WEXITSTATUS 0 Nov 05 00:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_538(8159599) done Nov 05 00:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907436 WEXITSTATUS 0 Nov 05 00:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907436 done Nov 05 00:01:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_641(7572810) Nov 05 00:01:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1995(8125622) Nov 05 00:01:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8151655 Nov 05 00:01:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_545(8159606) WEXITSTATUS 0 Nov 05 00:01:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_545(8159606) done Nov 05 00:01:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2308(8152225) WEXITSTATUS 0 Nov 05 00:01:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2308(8152225) done Nov 05 00:01:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5436(8159560) WEXITSTATUS 1 Nov 05 00:01:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5436(8159560) done Nov 05 00:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157498 WEXITSTATUS 0 Nov 05 00:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157498 done Nov 05 00:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_551(8159612) WEXITSTATUS 0 Nov 05 00:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_551(8159612) done Nov 05 00:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5528(8160707) WEXITSTATUS 1 Nov 05 00:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5528(8160707) done Nov 05 00:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_320(8160391) WEXITSTATUS 0 Nov 05 00:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_320(8160391) done Nov 05 00:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1277(8135232) WEXITSTATUS 0 Nov 05 00:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1277(8135232) done Nov 05 00:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_543(8159604) WEXITSTATUS 0 Nov 05 00:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_543(8159604) done Nov 05 00:01:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_515(8159576) WEXITSTATUS 0 Nov 05 00:01:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_515(8159576) done Nov 05 00:01:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_5(8158579) WEXITSTATUS 0 Nov 05 00:01:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_5(8158579) done Nov 05 00:01:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_456(8159498) WEXITSTATUS 0 Nov 05 00:01:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_456(8159498) done Nov 05 00:01:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_27(8155335) Nov 05 00:01:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_28(8155336) Nov 05 00:01:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_29(8155337) Nov 05 00:01:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_30(8155338) Nov 05 00:01:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_564(8159625) WEXITSTATUS 0 Nov 05 00:01:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_564(8159625) done Nov 05 00:01:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_521(8159582) WEXITSTATUS 0 Nov 05 00:01:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_521(8159582) done Nov 05 00:01:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_541(8159602) WEXITSTATUS 0 Nov 05 00:01:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_541(8159602) done Nov 05 00:01:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_567(8159628) WEXITSTATUS 0 Nov 05 00:01:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_567(8159628) done Nov 05 00:01:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_526(8159587) WEXITSTATUS 0 Nov 05 00:01:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_526(8159587) done Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:01:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_15(8158589) WEXITSTATUS 0 Nov 05 00:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_15(8158589) done Nov 05 00:02:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_540(8159601) WEXITSTATUS 0 Nov 05 00:02:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_540(8159601) done Nov 05 00:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_531(8159592) WEXITSTATUS 0 Nov 05 00:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_531(8159592) done Nov 05 00:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_513(8159574) WEXITSTATUS 0 Nov 05 00:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_513(8159574) done Nov 05 00:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_542(8159603) WEXITSTATUS 0 Nov 05 00:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_542(8159603) done Nov 05 00:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3578(8159262) WEXITSTATUS 0 Nov 05 00:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3578(8159262) done Nov 05 00:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3615(8161396) WEXITSTATUS 0 Nov 05 00:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3615(8161396) done Nov 05 00:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_539(8159600) WEXITSTATUS 0 Nov 05 00:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_539(8159600) done Nov 05 00:02:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_458(8159500) WEXITSTATUS 0 Nov 05 00:02:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_458(8159500) done Nov 05 00:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157503 WEXITSTATUS 0 Nov 05 00:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157503 done Nov 05 00:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_19(8158593) WEXITSTATUS 0 Nov 05 00:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_19(8158593) done Nov 05 00:02:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:02:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:02:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_1(8159093) WEXITSTATUS 0 Nov 05 00:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_1(8159093) done Nov 05 00:02:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_2(8159094) WEXITSTATUS 0 Nov 05 00:02:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_2(8159094) done Nov 05 00:02:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158970 Nov 05 00:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_534(8159595) WEXITSTATUS 0 Nov 05 00:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_534(8159595) done Nov 05 00:02:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2266(8152183) WEXITSTATUS 0 Nov 05 00:02:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2266(8152183) done Nov 05 00:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_536(8159597) WEXITSTATUS 0 Nov 05 00:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_536(8159597) done Nov 05 00:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5417(8159462) WEXITSTATUS 1 Nov 05 00:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5417(8159462) done Nov 05 00:02:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_550(8159611) WEXITSTATUS 0 Nov 05 00:02:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_550(8159611) done Nov 05 00:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_553(8159614) WEXITSTATUS 0 Nov 05 00:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_553(8159614) done Nov 05 00:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_460(8159502) WEXITSTATUS 0 Nov 05 00:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_460(8159502) done Nov 05 00:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3614(8161395) WEXITSTATUS 0 Nov 05 00:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3614(8161395) done Nov 05 00:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_546(8159607) WEXITSTATUS 0 Nov 05 00:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_546(8159607) done Nov 05 00:03:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_20(8158594) WEXITSTATUS 0 Nov 05 00:03:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_20(8158594) done Nov 05 00:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_25(8158625) WEXITSTATUS 0 Nov 05 00:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_25(8158625) done Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_612(8161409) in pi_ohern on r917u13n01 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_613(8161410) in pi_ohern on r917u13n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_614(8161411) in pi_ohern on r917u13n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_615(8161412) in pi_ohern on r917u13n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_616(8161413) in pi_ohern on r917u13n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_617(8161414) in pi_ohern on r917u13n04 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_618(8161415) in pi_ohern on r917u13n04 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_619(8161416) in pi_ohern on r917u13n04 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_620(8161417) in pi_ohern on r917u29n01 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_621(8161418) in pi_ohern on r917u29n01 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_622(8161419) in pi_ohern on r917u29n01 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_623(8161420) in pi_ohern on r917u29n01 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_624(8161421) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_625(8161422) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_626(8161423) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_627(8161424) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_628(8161425) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_629(8161426) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_630(8161427) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_631(8161428) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_632(8161429) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_633(8161430) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_634(8161431) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_635(8161432) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_636(8161433) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_637(8161434) in pi_ohern on r917u29n02 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_638(8161435) in pi_ohern on r917u29n03 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_639(8161436) in pi_ohern on r917u29n03 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_640(8161437) in pi_ohern on r917u29n03 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_641(8161438) in pi_ohern on r917u29n03 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_642(8161439) in pi_ohern on r917u29n04 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_643(8161440) in pi_ohern on r917u29n04 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_644(8161441) in pi_ohern on r903u12n01 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_645(8161442) in pi_ohern on r807u22n01 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_646(8161443) in pi_ohern on r807u24n01 Nov 05 00:03:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159911 in day on r815u09n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_800(8161444) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_801(8161445) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_802(8161446) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_803(8161447) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_804(8161448) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_805(8161449) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_806(8161450) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_807(8161451) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_808(8161452) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_809(8161453) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_810(8161454) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_811(8161455) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_812(8161456) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_813(8161457) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_814(8161458) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_815(8161459) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_816(8161460) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_817(8161461) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_818(8161462) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_819(8161463) in day on r807u11n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_820(8161464) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_821(8161465) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_822(8161466) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_823(8161467) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_824(8161468) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_825(8161469) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_826(8161470) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_827(8161471) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_828(8161472) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_829(8161473) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_830(8161474) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_831(8161475) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_832(8161476) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_833(8161477) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_834(8161478) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_835(8161479) in day on r909u01n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_836(8161480) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_837(8161481) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_838(8161482) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_839(8161483) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_840(8161484) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_841(8161485) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_842(8161486) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_843(8161487) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_844(8161488) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_845(8161489) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_846(8161490) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_847(8161491) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_848(8161492) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_849(8161493) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_850(8161494) in day on r912u28n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_851(8161495) in day on r912u36n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_852(8161496) in day on r912u36n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_853(8161497) in day on r918u05n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_854(8161498) in day on r919u36n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_855(8161499) in day on r919u36n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_856(8161500) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_857(8161501) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_858(8161502) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_859(8161503) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_860(8161504) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_861(8161505) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_862(8161506) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_863(8161507) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_864(8161508) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_865(8161509) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_866(8161510) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_867(8161511) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_868(8161512) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_869(8161513) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_870(8161514) in day on r805u04n04 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_871(8161515) in day on r805u06n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_872(8161516) in day on r805u06n02 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_873(8161517) in day on r805u06n03 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_874(8161518) in day on r805u06n03 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_875(8161519) in day on r805u06n03 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_876(8161520) in day on r805u06n03 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_877(8161521) in day on r805u06n03 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_878(8161522) in day on r805u06n03 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_879(8161523) in day on r805u06n03 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_880(8161524) in day on r805u06n03 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_881(8161525) in day on r805u14n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_882(8161526) in day on r805u14n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_883(8161527) in day on r805u14n01 Nov 05 00:03:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_884(8161528) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_885(8161529) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_886(8161530) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_887(8161531) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_888(8161532) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_889(8161533) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_890(8161534) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_891(8161535) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_892(8161536) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_893(8161537) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_894(8161538) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_895(8161539) in day on r805u14n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_896(8161540) in day on r805u23n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_897(8161541) in day on r805u23n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_898(8161542) in day on r805u23n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_899(8161543) in day on r805u23n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_27(8161544) in day on r805u23n01 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_28(8161545) in day on r805u25n04 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_29(8161546) in day on r805u27n02 Nov 05 00:03:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907437 in gpu on r805u38n01 Nov 05 00:03:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3618(8161547) in gpu on r903u40n01 Nov 05 00:03:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3619(8161548) in gpu on r808u08n01 Nov 05 00:03:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3620(8161549) in gpu on r808u14n01 Nov 05 00:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5587(8161407) WEXITSTATUS 1 Nov 05 00:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5587(8161407) done Nov 05 00:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_722(7572891) WEXITSTATUS 0 Nov 05 00:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_722(7572891) done Nov 05 00:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157502 WEXITSTATUS 0 Nov 05 00:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157502 done Nov 05 00:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_777(7572946) WEXITSTATUS 0 Nov 05 00:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_777(7572946) done Nov 05 00:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5482(8160131) WEXITSTATUS 1 Nov 05 00:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5482(8160131) done Nov 05 00:03:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_557(8159618) WEXITSTATUS 0 Nov 05 00:03:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_557(8159618) done Nov 05 00:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_556(8159617) WEXITSTATUS 0 Nov 05 00:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_556(8159617) done Nov 05 00:03:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r805u23n01 Nov 05 00:03:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_532(8159593) WEXITSTATUS 0 Nov 05 00:03:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_532(8159593) done Nov 05 00:03:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8161550 NodeList=(null) usec=398048 Nov 05 00:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5588(8161551) in scavenge on r801u11n04 Nov 05 00:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5589(8161552) in scavenge on r911u28n01 Nov 05 00:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5590(8161553) in scavenge on r816u23n11 Nov 05 00:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5591(8161554) in scavenge on r816u23n11 Nov 05 00:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5592(8161555) in scavenge on r816u23n11 Nov 05 00:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5593(8161556) in scavenge on r816u15n06 Nov 05 00:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5594(8161557) in scavenge on r816u35n03 Nov 05 00:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3594(8159545) WEXITSTATUS 0 Nov 05 00:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3594(8159545) done Nov 05 00:04:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1868(8161558) in scavenge on r816u23n11 Nov 05 00:04:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1869(8161559) in scavenge on r816u23n11 Nov 05 00:04:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1870(8161560) in scavenge on r816u23n11 Nov 05 00:04:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1871(8161561) in scavenge on r816u23n11 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:04:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_7(8158581) WEXITSTATUS 0 Nov 05 00:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_7(8158581) done Nov 05 00:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158541 WEXITSTATUS 0 Nov 05 00:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158541 done Nov 05 00:04:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5090(8158414) WEXITSTATUS 1 Nov 05 00:04:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5090(8158414) done Nov 05 00:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5331(8158904) WEXITSTATUS 1 Nov 05 00:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5331(8158904) done Nov 05 00:04:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_555(8159616) WEXITSTATUS 0 Nov 05 00:04:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_555(8159616) done Nov 05 00:04:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5567(8161221) WEXITSTATUS 1 Nov 05 00:04:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5567(8161221) done Nov 05 00:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5554(8161044) WEXITSTATUS 1 Nov 05 00:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5554(8161044) done Nov 05 00:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_533(8159594) WEXITSTATUS 0 Nov 05 00:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_533(8159594) done Nov 05 00:05:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158542 WEXITSTATUS 0 Nov 05 00:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158542 done Nov 05 00:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3619(8161548) WEXITSTATUS 0 Nov 05 00:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3619(8161548) done Nov 05 00:05:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_452(8159494) WEXITSTATUS 0 Nov 05 00:05:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_452(8159494) done Nov 05 00:05:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5565(8161219) WEXITSTATUS 1 Nov 05 00:05:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5565(8161219) done Nov 05 00:05:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_9(8158583) WEXITSTATUS 0 Nov 05 00:05:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_9(8158583) done Nov 05 00:05:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158543 WEXITSTATUS 0 Nov 05 00:05:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158543 done Nov 05 00:06:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157501 WEXITSTATUS 0 Nov 05 00:06:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157501 done Nov 05 00:06:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159112 WEXITSTATUS 0 Nov 05 00:06:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159112 done Nov 05 00:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5525(8160704) WEXITSTATUS 1 Nov 05 00:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5525(8160704) done Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 00:06:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 00:06:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8156213 Nov 05 00:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_8(8158582) WEXITSTATUS 0 Nov 05 00:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_8(8158582) done Nov 05 00:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157500 WEXITSTATUS 0 Nov 05 00:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157500 done Nov 05 00:06:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161562 InitPrio=61739 usec=4878 Nov 05 00:06:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161562 NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 00:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161562 WEXITSTATUS 127 Nov 05 00:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161562 done Nov 05 00:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_12(8158586) WEXITSTATUS 0 Nov 05 00:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_12(8158586) done Nov 05 00:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161550 in pi_balou on r802u30n04 Nov 05 00:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_647(8161563) in pi_ohern on r917u29n01 Nov 05 00:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_648(8161564) in pi_ohern on r917u29n01 Nov 05 00:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_649(8161565) in pi_ohern on r917u29n02 Nov 05 00:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_650(8161566) in pi_ohern on r917u29n04 Nov 05 00:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_651(8161567) in pi_ohern on r917u29n04 Nov 05 00:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_652(8161568) in pi_ohern on r917u29n04 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_900(8161569) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_901(8161570) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_902(8161571) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_903(8161572) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_904(8161573) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_905(8161574) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_906(8161575) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_907(8161576) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_908(8161577) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_909(8161578) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_910(8161579) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_911(8161580) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_912(8161581) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_913(8161582) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_914(8161583) in day on r806u02n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_915(8161584) in day on r909u01n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_916(8161585) in day on r909u01n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_917(8161586) in day on r909u01n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_918(8161587) in day on r919u36n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_919(8161588) in day on r919u36n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_920(8161589) in day on r919u36n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_921(8161590) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_922(8161591) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_923(8161592) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_924(8161593) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_925(8161594) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_926(8161595) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_927(8161596) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_928(8161597) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_929(8161598) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_930(8161599) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_931(8161600) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_932(8161601) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_933(8161602) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_934(8161603) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_935(8161604) in day on r912u28n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_936(8161605) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_937(8161606) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_938(8161607) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_939(8161608) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_940(8161609) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_941(8161610) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_942(8161611) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_943(8161612) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_944(8161613) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_945(8161614) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_946(8161615) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_947(8161616) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_948(8161617) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_949(8161618) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_950(8161619) in day on r913u28n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_951(8161620) in day on r805u02n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_952(8161621) in day on r805u02n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_953(8161622) in day on r805u02n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_954(8161623) in day on r805u06n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_955(8161624) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_956(8161625) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_957(8161626) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_958(8161627) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_959(8161628) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_960(8161629) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_961(8161630) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_962(8161631) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_963(8161632) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_964(8161633) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_965(8161634) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_966(8161635) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_967(8161636) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_968(8161637) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_969(8161638) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_970(8161639) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_971(8161640) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_972(8161641) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_973(8161642) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_974(8161643) in day on r805u09n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_975(8161644) in day on r805u23n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_976(8161645) in day on r815u23n05 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_977(8161646) in day on r815u23n06 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_978(8161647) in day on r815u29n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_979(8161648) in day on r815u29n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_980(8161649) in day on r815u29n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_981(8161650) in day on r815u29n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_982(8161651) in day on r815u29n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_983(8161652) in day on r815u29n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_984(8161653) in day on r815u29n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_985(8161654) in day on r815u29n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_986(8161655) in day on r815u29n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_987(8161656) in day on r815u29n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_988(8161657) in day on r815u29n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_989(8161658) in day on r815u35n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_990(8161659) in day on r815u35n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_991(8161660) in day on r815u35n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_992(8161661) in day on r815u35n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_993(8161662) in day on r815u35n02 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_994(8161663) in day on r815u35n03 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_995(8161664) in day on r815u35n04 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_996(8161665) in day on r815u35n04 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_997(8161666) in day on r815u35n04 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_998(8161667) in day on r815u35n04 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159941_999(8159941) in day on r815u35n05 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_30(8161668) in day on r815u09n01 Nov 05 00:06:38 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:206 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 00:06:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_829(7571795) Nov 05 00:06:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3621(8161669) in gpu on r808u08n01 Nov 05 00:06:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3622(8161670) in gpu on r808u11n01 Nov 05 00:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_821(7572990) WEXITSTATUS 0 Nov 05 00:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_821(7572990) done Nov 05 00:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_887(7571853) WEXITSTATUS 0 Nov 05 00:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_887(7571853) done Nov 05 00:06:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5510(8160513) WEXITSTATUS 1 Nov 05 00:06:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5510(8160513) done Nov 05 00:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5410(8159391) WEXITSTATUS 1 Nov 05 00:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5410(8159391) done Nov 05 00:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5595(8161671) in scavenge on r907u01n04 Nov 05 00:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5596(8161672) in scavenge on r907u05n02 Nov 05 00:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5597(8161673) in scavenge on r802u06n02 Nov 05 00:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5598(8161674) in scavenge on r816u35n04 Nov 05 00:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4612(8156191) WEXITSTATUS 1 Nov 05 00:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4612(8156191) done Nov 05 00:07:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5270(8158704) WEXITSTATUS 1 Nov 05 00:07:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5270(8158704) done Nov 05 00:07:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161675 InitPrio=29688 usec=7107 Nov 05 00:07:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161676 InitPrio=29688 usec=5248 Nov 05 00:07:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158544 WEXITSTATUS 0 Nov 05 00:07:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158544 done Nov 05 00:07:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161677 InitPrio=29688 usec=4486 Nov 05 00:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5582(8161402) WEXITSTATUS 1 Nov 05 00:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5582(8161402) done Nov 05 00:07:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_10(8159118) WEXITSTATUS 0 Nov 05 00:07:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_10(8159118) done Nov 05 00:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1979(8151417) WEXITSTATUS 0 Nov 05 00:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1979(8151417) done Nov 05 00:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_548(8159609) WEXITSTATUS 0 Nov 05 00:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_548(8159609) done Nov 05 00:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3622(8161670) WEXITSTATUS 0 Nov 05 00:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3622(8161670) done Nov 05 00:07:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161678 InitPrio=21238 usec=8099 Nov 05 00:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_8(8159189) WEXITSTATUS 0 Nov 05 00:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_8(8159189) done Nov 05 00:08:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158710 OOM failure Nov 05 00:08:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158710 done Nov 05 00:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157507 WEXITSTATUS 0 Nov 05 00:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157507 done Nov 05 00:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_8(8159100) WEXITSTATUS 0 Nov 05 00:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_8(8159100) done Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3620(8161549) WEXITSTATUS 0 Nov 05 00:08:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3620(8161549) done Nov 05 00:08:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1416(8135949) WEXITSTATUS 0 Nov 05 00:08:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1416(8135949) done Nov 05 00:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4957(8158207) WEXITSTATUS 1 Nov 05 00:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4957(8158207) done Nov 05 00:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_7(8159099) WEXITSTATUS 0 Nov 05 00:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_7(8159099) done Nov 05 00:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_26(8158640) WEXITSTATUS 0 Nov 05 00:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_26(8158640) done Nov 05 00:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_6(8159098) WEXITSTATUS 0 Nov 05 00:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_6(8159098) done Nov 05 00:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2219(8152011) WEXITSTATUS 0 Nov 05 00:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2219(8152011) done Nov 05 00:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157508 WEXITSTATUS 0 Nov 05 00:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157508 done Nov 05 00:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5162(8158516) WEXITSTATUS 1 Nov 05 00:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5162(8158516) done Nov 05 00:09:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_653(8161679) in pi_ohern on r917u29n02 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_31(8161680) in day on r805u27n03 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_32(8161681) in day on r806u06n03 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_33(8161682) in day on r807u05n01 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_965(7573134) WEXITSTATUS 0 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_965(7573134) done Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_899(7573068) WEXITSTATUS 0 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_899(7573068) done Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_553(7572722) WEXITSTATUS 0 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_553(7572722) done Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_633(7572802) WEXITSTATUS 0 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_633(7572802) done Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1010(7573179) WEXITSTATUS 0 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1010(7573179) done Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_960(7573129) WEXITSTATUS 0 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_960(7573129) done Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_690(7572859) WEXITSTATUS 0 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_690(7572859) done Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_916(7573085) WEXITSTATUS 0 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_916(7573085) done Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3623(8161683) in gpu on r808u11n01 Nov 05 00:09:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3624(8161684) in gpu on r808u14n01 Nov 05 00:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5178(8158532) WEXITSTATUS 1 Nov 05 00:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5178(8158532) done Nov 05 00:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157510 WEXITSTATUS 0 Nov 05 00:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157510 done Nov 05 00:09:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r805u27n03 Nov 05 00:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157509 WEXITSTATUS 0 Nov 05 00:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157509 done Nov 05 00:09:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5088(8158412) WEXITSTATUS 1 Nov 05 00:09:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5088(8158412) done Nov 05 00:09:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161685 InitPrio=40248 usec=4473 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5599(8161686) in scavenge on r802u02n03 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5600(8161687) in scavenge on r903u07n01 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5601(8161688) in scavenge on r816u23n08 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5602(8161689) in scavenge on r816u23n12 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5603(8161690) in scavenge on r907u28n01 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5604(8161691) in scavenge on r912u36n01 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5605(8161692) in scavenge on r913u09n01 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5606(8161693) in scavenge on r913u09n01 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5607(8161694) in scavenge on r815u15n04 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5608(8161695) in scavenge on r815u15n04 Nov 05 00:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5609(8161696) in scavenge on r816u29n02 Nov 05 00:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3618(8161547) WEXITSTATUS 0 Nov 05 00:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3618(8161547) done Nov 05 00:10:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8161685 uid 20398 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5530(8160865) WEXITSTATUS 1 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5530(8160865) done Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5362(8159172) WEXITSTATUS 1 Nov 05 00:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5362(8159172) done Nov 05 00:10:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_549(8159610) WEXITSTATUS 0 Nov 05 00:10:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158777_549(8159610) done Nov 05 00:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4259(8153354) WEXITSTATUS 1 Nov 05 00:10:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4259(8153354) done Nov 05 00:10:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8161236 uid 20398 Nov 05 00:10:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161697 InitPrio=40248 usec=4393 Nov 05 00:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161697 NodeList=r807u03n02 #CPUs=8 Partition=day Nov 05 00:10:39 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2924(8128608) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:10:39 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2932(8128616) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:10:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2924(8128608) Nov 05 00:10:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2932(8128616) Nov 05 00:10:43 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 00:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5591(8161554) WEXITSTATUS 1 Nov 05 00:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5591(8161554) done Nov 05 00:10:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154085 WEXITSTATUS 0 Nov 05 00:10:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154085 done Nov 05 00:10:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161698 InitPrio=59533 usec=3899 Nov 05 00:10:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161699 InitPrio=59533 usec=4308 Nov 05 00:10:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161700 InitPrio=59533 usec=3982 Nov 05 00:10:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161701 InitPrio=59533 usec=3655 Nov 05 00:10:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161698 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:10:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161699 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:10:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161700 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:10:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161701 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:11:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_3(8159746) WEXITSTATUS 0 Nov 05 00:11:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_3(8159746) done Nov 05 00:11:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_12(8159755) WEXITSTATUS 0 Nov 05 00:11:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_12(8159755) done Nov 05 00:11:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_2(8159183) WEXITSTATUS 0 Nov 05 00:11:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_2(8159183) done Nov 05 00:11:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_9(8159190) WEXITSTATUS 0 Nov 05 00:11:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_9(8159190) done Nov 05 00:11:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_3(8159184) WEXITSTATUS 0 Nov 05 00:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_3(8159184) done Nov 05 00:11:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4765(8157336) WEXITSTATUS 1 Nov 05 00:11:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4765(8157336) done Nov 05 00:11:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1414(8135947) WEXITSTATUS 0 Nov 05 00:11:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1414(8135947) done Nov 05 00:11:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161699 WEXITSTATUS 0 Nov 05 00:11:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161699 done Nov 05 00:11:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161698 WEXITSTATUS 0 Nov 05 00:11:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161698 done Nov 05 00:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161701 WEXITSTATUS 0 Nov 05 00:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161701 done Nov 05 00:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161700 WEXITSTATUS 0 Nov 05 00:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161700 done Nov 05 00:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_4(8159185) WEXITSTATUS 0 Nov 05 00:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_4(8159185) done Nov 05 00:11:42 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8161702 NodeList=(null) usec=27489 Nov 05 00:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5590(8161553) WEXITSTATUS 1 Nov 05 00:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5590(8161553) done Nov 05 00:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5544(8160879) WEXITSTATUS 1 Nov 05 00:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5544(8160879) done Nov 05 00:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4440(8155681) WEXITSTATUS 1 Nov 05 00:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4440(8155681) done Nov 05 00:11:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5593(8161556) WEXITSTATUS 1 Nov 05 00:11:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5593(8161556) done Nov 05 00:11:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1918(8151356) WEXITSTATUS 0 Nov 05 00:11:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1918(8151356) done Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_38(8158686) WEXITSTATUS 0 Nov 05 00:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_38(8158686) done Nov 05 00:12:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_42(8135741) WEXITSTATUS 0 Nov 05 00:12:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_42(8135741) done Nov 05 00:12:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_31(8155683) Nov 05 00:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161702 in pi_balou on r802u30n04 Nov 05 00:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_654(8161703) in pi_ohern on r917u29n03 Nov 05 00:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_655(8161704) in pi_ohern on r903u26n01 Nov 05 00:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_656(8161705) in pi_ohern on r903u26n01 Nov 05 00:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161678 in day on r909u29n03,r918u09n04 Nov 05 00:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_34(8161706) in day on r805u06n04 Nov 05 00:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_35(8161707) in day on r807u03n02 Nov 05 00:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_36(8161708) in day on r913u24n03 Nov 05 00:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_37(8161709) in day on r904u10n02 Nov 05 00:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3625(8161710) in gpu on r903u40n01 Nov 05 00:12:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_127(8114393) WEXITSTATUS 0 Nov 05 00:12:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_127(8114393) done Nov 05 00:12:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_851(7571817) WEXITSTATUS 0 Nov 05 00:12:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_851(7571817) done Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_332(8114599) WEXITSTATUS 0 Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_332(8114599) done Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_7(8114273) WEXITSTATUS 0 Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_7(8114273) done Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_655(8161704) WEXITSTATUS 0 Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_655(8161704) done Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_656(8161705) WEXITSTATUS 0 Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_656(8161705) done Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4293(8154059) WEXITSTATUS 1 Nov 05 00:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4293(8154059) done Nov 05 00:12:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5311(8158773) WEXITSTATUS 1 Nov 05 00:12:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5311(8158773) done Nov 05 00:12:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5539(8160874) WEXITSTATUS 1 Nov 05 00:12:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5539(8160874) done Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5610(8161711) in scavenge on r903u01n01 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5611(8161712) in scavenge on r903u04n04 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5612(8161713) in scavenge on r815u23n07 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5613(8161714) in scavenge on r816u09n10 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5614(8161715) in scavenge on r816u23n11 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5615(8161716) in scavenge on r816u23n11 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5616(8161717) in scavenge on r914u01n01 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5617(8161718) in scavenge on r919u36n01 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5618(8161719) in scavenge on r909u04n04 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5619(8161720) in scavenge on r909u07n04 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5620(8161721) in scavenge on r816u15n06 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5621(8161722) in scavenge on r815u23n06 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5622(8161723) in scavenge on r815u23n06 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5623(8161724) in scavenge on r815u23n06 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5624(8161725) in scavenge on r815u29n01 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_14(8159757) WEXITSTATUS 0 Nov 05 00:12:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_14(8159757) done Nov 05 00:12:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_457(7571422) in scavenge on r907u05n01 Nov 05 00:12:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_173(7572341) in scavenge on r815u09n09 Nov 05 00:13:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5308(8158770) WEXITSTATUS 1 Nov 05 00:13:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5308(8158770) done Nov 05 00:13:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=62265573 NodeList=r908u33n01 #CPUs=1 Partition=admintest Nov 05 00:13:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5358(8159092) WEXITSTATUS 1 Nov 05 00:13:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5358(8159092) done Nov 05 00:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1872(8161726) in scavenge on r802u23n02 Nov 05 00:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_44(8158714) WEXITSTATUS 0 Nov 05 00:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_44(8158714) done Nov 05 00:13:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5506(8160509) WEXITSTATUS 1 Nov 05 00:13:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5506(8160509) done Nov 05 00:13:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=62265573 WEXITSTATUS 0 Nov 05 00:13:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=62265573 done Nov 05 00:13:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=62265573 Nov 05 00:13:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161727 InitPrio=13461 usec=4139 Nov 05 00:13:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_2(8159745) WEXITSTATUS 0 Nov 05 00:13:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_2(8159745) done Nov 05 00:13:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158297 WEXITSTATUS 0 Nov 05 00:13:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158297 done Nov 05 00:13:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2295(8152212) WEXITSTATUS 0 Nov 05 00:13:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2295(8152212) done Nov 05 00:13:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_36(8159804) WEXITSTATUS 0 Nov 05 00:13:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_36(8159804) done Nov 05 00:13:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_68(8159836) WEXITSTATUS 0 Nov 05 00:13:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_68(8159836) done Nov 05 00:13:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_138(8159966) WEXITSTATUS 0 Nov 05 00:13:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_138(8159966) done Nov 05 00:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5228(8158631) WEXITSTATUS 1 Nov 05 00:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5228(8158631) done Nov 05 00:13:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_10(8159753) WEXITSTATUS 0 Nov 05 00:13:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_10(8159753) done Nov 05 00:13:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5102(8158435) WEXITSTATUS 1 Nov 05 00:13:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5102(8158435) done Nov 05 00:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_1(8159744) WEXITSTATUS 0 Nov 05 00:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_1(8159744) done Nov 05 00:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_11(8159754) WEXITSTATUS 0 Nov 05 00:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_11(8159754) done Nov 05 00:14:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_25(7570988) Nov 05 00:14:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1041(7572007) Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:14:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:14:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_140(8159968) WEXITSTATUS 0 Nov 05 00:14:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_140(8159968) done Nov 05 00:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_141(8159969) WEXITSTATUS 0 Nov 05 00:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_141(8159969) done Nov 05 00:14:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5455(8159771) WEXITSTATUS 1 Nov 05 00:14:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5455(8159771) done Nov 05 00:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_80(8159848) WEXITSTATUS 0 Nov 05 00:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_80(8159848) done Nov 05 00:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_127(8159955) WEXITSTATUS 0 Nov 05 00:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_127(8159955) done Nov 05 00:14:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3674(8138147) WEXITSTATUS 1 Nov 05 00:14:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3674(8138147) done Nov 05 00:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1257(8135160) WEXITSTATUS 0 Nov 05 00:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1257(8135160) done Nov 05 00:14:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8152490 Nov 05 00:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5073(8158385) WEXITSTATUS 1 Nov 05 00:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5073(8158385) done Nov 05 00:14:44 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:44 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:46 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:48 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161728 InitPrio=13461 usec=3743 Nov 05 00:14:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:50 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_83(8159851) WEXITSTATUS 0 Nov 05 00:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_83(8159851) done Nov 05 00:14:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_4(8159747) WEXITSTATUS 0 Nov 05 00:14:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_4(8159747) done Nov 05 00:15:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_81(8159849) WEXITSTATUS 0 Nov 05 00:15:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_81(8159849) done Nov 05 00:15:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=62265572 NodeList=r908u33n01 #CPUs=1 Partition=admintest Nov 05 00:15:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_130(8159958) WEXITSTATUS 0 Nov 05 00:15:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_130(8159958) done Nov 05 00:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5604(8161691) WEXITSTATUS 1 Nov 05 00:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5604(8161691) done Nov 05 00:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=62265572 WEXITSTATUS 0 Nov 05 00:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=62265572 done Nov 05 00:15:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=62265572 Nov 05 00:15:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1444(8136303) WEXITSTATUS 0 Nov 05 00:15:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1444(8136303) done Nov 05 00:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_136(8159964) WEXITSTATUS 0 Nov 05 00:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_136(8159964) done Nov 05 00:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_8(8159751) WEXITSTATUS 0 Nov 05 00:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_8(8159751) done Nov 05 00:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_46(8159814) WEXITSTATUS 0 Nov 05 00:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_46(8159814) done Nov 05 00:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_39(8158687) WEXITSTATUS 0 Nov 05 00:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_39(8158687) done Nov 05 00:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_78(8159846) WEXITSTATUS 0 Nov 05 00:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_78(8159846) done Nov 05 00:15:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_32(8155772) Nov 05 00:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_132(8159960) WEXITSTATUS 0 Nov 05 00:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_132(8159960) done Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161727 in pi_balou on r802u30n04 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161728 in pi_balou on r802u30n04 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_657(8161729) in pi_ohern on r917u13n02 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_658(8161730) in pi_ohern on r917u13n02 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_659(8161731) in pi_ohern on r917u13n03 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_660(8161732) in pi_ohern on r815u03n12 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_661(8161733) in pi_ohern on r815u03n12 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_662(8161734) in pi_ohern on r903u12n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_663(8161735) in pi_ohern on r903u24n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_664(8161736) in pi_ohern on r903u26n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_665(8161737) in pi_ohern on r903u26n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_666(8161738) in pi_ohern on r903u26n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_667(8161739) in pi_ohern on r903u26n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158906_668(8158906) in pi_ohern on r903u26n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_1(8161740) in pi_ohern on r903u26n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_2(8161741) in pi_ohern on r903u26n01 Nov 05 00:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159912 in day on r815u35n03 Nov 05 00:15:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_38(8161742) in day on r904u04n02 Nov 05 00:15:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_39(8161743) in day on r805u02n01 Nov 05 00:15:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_40(8161744) in day on r805u14n02 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_657(8161729) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_657(8161729) done Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1030(7573199) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1030(7573199) done Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_407(8114674) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_407(8114674) done Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_667(8161739) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_667(8161739) done Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_664(8161736) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_664(8161736) done Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_666(8161738) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_666(8161738) done Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_668(8158906) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_668(8158906) done Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_665(8161737) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_665(8161737) done Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_658(8161730) WEXITSTATUS 0 Nov 05 00:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_658(8161730) done Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_663(8161735) WEXITSTATUS 0 Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_663(8161735) done Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_661(8161733) WEXITSTATUS 0 Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_661(8161733) done Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_659(8161731) WEXITSTATUS 0 Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_659(8161731) done Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_662(8161734) WEXITSTATUS 0 Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_662(8161734) done Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_660(8161732) WEXITSTATUS 0 Nov 05 00:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_660(8161732) done Nov 05 00:15:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r806u14n04 Nov 05 00:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_137(8159965) WEXITSTATUS 0 Nov 05 00:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_137(8159965) done Nov 05 00:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4849(8157857) WEXITSTATUS 1 Nov 05 00:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4849(8157857) done Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5625(8161745) in scavenge on r907u05n01 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5626(8161746) in scavenge on r802u09n02 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5627(8161747) in scavenge on r915u32n01 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5628(8161748) in scavenge on r805u06n03 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5629(8161749) in scavenge on r806u14n04 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5630(8161750) in scavenge on r807u08n03 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5631(8161751) in scavenge on r909u07n04 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5632(8161752) in scavenge on r904u04n02 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5633(8161753) in scavenge on r816u29n01 Nov 05 00:15:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5634(8161754) in scavenge on r816u29n02 Nov 05 00:15:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_9(8159752) WEXITSTATUS 0 Nov 05 00:15:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_9(8159752) done Nov 05 00:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5546(8161036) WEXITSTATUS 1 Nov 05 00:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5546(8161036) done Nov 05 00:16:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1873(8161755) in scavenge on r816u29n01 Nov 05 00:16:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1874(8161756) in scavenge on r816u35n03 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8159276 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:16:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_7(8159750) WEXITSTATUS 0 Nov 05 00:16:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_7(8159750) done Nov 05 00:16:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1498(8142111) WEXITSTATUS 0 Nov 05 00:16:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1498(8142111) done Nov 05 00:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_79(8159847) WEXITSTATUS 0 Nov 05 00:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_79(8159847) done Nov 05 00:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2344(8152261) WEXITSTATUS 0 Nov 05 00:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2344(8152261) done Nov 05 00:16:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161757 InitPrio=59533 usec=4585 Nov 05 00:16:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161758 InitPrio=59533 usec=3125 Nov 05 00:16:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161759 InitPrio=59533 usec=2623 Nov 05 00:16:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161760 InitPrio=59533 usec=4412 Nov 05 00:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_134(8159962) WEXITSTATUS 0 Nov 05 00:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_134(8159962) done Nov 05 00:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_131(8159959) WEXITSTATUS 0 Nov 05 00:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_131(8159959) done Nov 05 00:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5043(8158339) WEXITSTATUS 1 Nov 05 00:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5043(8158339) done Nov 05 00:16:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161757 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:16:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161758 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:16:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161759 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:16:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161760 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3624(8161684) WEXITSTATUS 0 Nov 05 00:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3624(8161684) done Nov 05 00:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_157(8159985) WEXITSTATUS 0 Nov 05 00:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_157(8159985) done Nov 05 00:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3600(8159653) WEXITSTATUS 0 Nov 05 00:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3600(8159653) done Nov 05 00:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2141(8125959) WEXITSTATUS 1 Nov 05 00:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2141(8125959) done Nov 05 00:16:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2013(8125773) Nov 05 00:16:41 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5339(8158958) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:16:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_55(8159823) WEXITSTATUS 0 Nov 05 00:16:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_55(8159823) done Nov 05 00:16:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5339(8158958) Nov 05 00:16:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_243(8160191) WEXITSTATUS 0 Nov 05 00:16:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_243(8160191) done Nov 05 00:16:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_6(8159749) WEXITSTATUS 0 Nov 05 00:16:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_6(8159749) done Nov 05 00:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_135(8159963) WEXITSTATUS 0 Nov 05 00:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_135(8159963) done Nov 05 00:16:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_48(8159816) WEXITSTATUS 0 Nov 05 00:16:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_48(8159816) done Nov 05 00:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_167(8159995) WEXITSTATUS 0 Nov 05 00:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_167(8159995) done Nov 05 00:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_52(8159820) WEXITSTATUS 0 Nov 05 00:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_52(8159820) done Nov 05 00:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157515 WEXITSTATUS 0 Nov 05 00:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157515 done Nov 05 00:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_166(8159994) WEXITSTATUS 0 Nov 05 00:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_166(8159994) done Nov 05 00:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_91(8159859) WEXITSTATUS 0 Nov 05 00:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_91(8159859) done Nov 05 00:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_33(8159801) WEXITSTATUS 0 Nov 05 00:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_33(8159801) done Nov 05 00:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_183(8160011) WEXITSTATUS 0 Nov 05 00:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_183(8160011) done Nov 05 00:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_176(8160004) WEXITSTATUS 0 Nov 05 00:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_176(8160004) done Nov 05 00:17:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_270(8160218) WEXITSTATUS 0 Nov 05 00:17:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_270(8160218) done Nov 05 00:17:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161757 WEXITSTATUS 0 Nov 05 00:17:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161757 done Nov 05 00:17:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161758 WEXITSTATUS 0 Nov 05 00:17:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161758 done Nov 05 00:17:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161759 WEXITSTATUS 0 Nov 05 00:17:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161759 done Nov 05 00:17:08 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:17:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161760 WEXITSTATUS 0 Nov 05 00:17:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161760 done Nov 05 00:17:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_158(8159986) WEXITSTATUS 0 Nov 05 00:17:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_158(8159986) done Nov 05 00:17:13 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1262(8135170) WEXITSTATUS 0 Nov 05 00:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1262(8135170) done Nov 05 00:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_252(8160200) WEXITSTATUS 0 Nov 05 00:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_252(8160200) done Nov 05 00:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_128(8159956) WEXITSTATUS 0 Nov 05 00:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_128(8159956) done Nov 05 00:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_139(8159967) WEXITSTATUS 0 Nov 05 00:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_139(8159967) done Nov 05 00:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2338(8152255) WEXITSTATUS 0 Nov 05 00:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2338(8152255) done Nov 05 00:17:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8161678 uid 21828 Nov 05 00:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5318(8158846) WEXITSTATUS 1 Nov 05 00:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5318(8158846) done Nov 05 00:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5180(8158534) WEXITSTATUS 1 Nov 05 00:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5180(8158534) done Nov 05 00:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_168(8159996) WEXITSTATUS 0 Nov 05 00:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_168(8159996) done Nov 05 00:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_43(8158713) WEXITSTATUS 0 Nov 05 00:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_43(8158713) done Nov 05 00:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_194(8160022) WEXITSTATUS 0 Nov 05 00:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_194(8160022) done Nov 05 00:17:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_44(8159812) WEXITSTATUS 0 Nov 05 00:17:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_44(8159812) done Nov 05 00:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_42(8159810) WEXITSTATUS 0 Nov 05 00:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_42(8159810) done Nov 05 00:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5477(8159900) WEXITSTATUS 1 Nov 05 00:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5477(8159900) done Nov 05 00:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_76(8159844) WEXITSTATUS 0 Nov 05 00:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_76(8159844) done Nov 05 00:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_178(8160006) WEXITSTATUS 0 Nov 05 00:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_178(8160006) done Nov 05 00:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_25(8159793) WEXITSTATUS 0 Nov 05 00:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_25(8159793) done Nov 05 00:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_22(8159790) WEXITSTATUS 0 Nov 05 00:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_22(8159790) done Nov 05 00:17:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_244(8160192) WEXITSTATUS 0 Nov 05 00:17:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_244(8160192) done Nov 05 00:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_251(8160199) WEXITSTATUS 0 Nov 05 00:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_251(8160199) done Nov 05 00:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4124(8149430) WEXITSTATUS 1 Nov 05 00:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4124(8149430) done Nov 05 00:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_63(8159831) WEXITSTATUS 0 Nov 05 00:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_63(8159831) done Nov 05 00:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_133(8159961) WEXITSTATUS 0 Nov 05 00:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_133(8159961) done Nov 05 00:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_84(8159852) WEXITSTATUS 0 Nov 05 00:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_84(8159852) done Nov 05 00:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5008(8158292) WEXITSTATUS 1 Nov 05 00:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5008(8158292) done Nov 05 00:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_82(8159850) WEXITSTATUS 0 Nov 05 00:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_82(8159850) done Nov 05 00:18:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:18:08 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:18:09 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:18:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:18:11 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:18:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:18:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_111(8159879) WEXITSTATUS 0 Nov 05 00:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_111(8159879) done Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:18:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:18:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154595 WEXITSTATUS 0 Nov 05 00:18:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154595 done Nov 05 00:18:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5628(8161748) WEXITSTATUS 1 Nov 05 00:18:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5628(8161748) done Nov 05 00:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_65(8159833) WEXITSTATUS 0 Nov 05 00:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_65(8159833) done Nov 05 00:18:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5428(8159552) WEXITSTATUS 1 Nov 05 00:18:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5428(8159552) done Nov 05 00:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_39(8159807) WEXITSTATUS 0 Nov 05 00:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_39(8159807) done Nov 05 00:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_56(8159824) WEXITSTATUS 0 Nov 05 00:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_56(8159824) done Nov 05 00:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_256(8160204) WEXITSTATUS 0 Nov 05 00:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_256(8160204) done Nov 05 00:18:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_77(8159845) WEXITSTATUS 0 Nov 05 00:18:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_77(8159845) done Nov 05 00:18:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_107(8159875) WEXITSTATUS 0 Nov 05 00:18:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_107(8159875) done Nov 05 00:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_13(8159756) WEXITSTATUS 0 Nov 05 00:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_13(8159756) done Nov 05 00:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_29(8159797) WEXITSTATUS 0 Nov 05 00:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_29(8159797) done Nov 05 00:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_57(8159825) WEXITSTATUS 0 Nov 05 00:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_57(8159825) done Nov 05 00:18:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_20(8159788) WEXITSTATUS 0 Nov 05 00:18:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_20(8159788) done Nov 05 00:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_259(8160207) WEXITSTATUS 0 Nov 05 00:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_259(8160207) done Nov 05 00:18:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_49(8159817) WEXITSTATUS 0 Nov 05 00:18:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_49(8159817) done Nov 05 00:18:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_47(8159815) WEXITSTATUS 0 Nov 05 00:18:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_47(8159815) done Nov 05 00:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8151449 WEXITSTATUS 0 Nov 05 00:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8151449 done Nov 05 00:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8153757 WEXITSTATUS 0 Nov 05 00:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8153757 done Nov 05 00:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_3(8161761) in pi_ohern on r801u32n02 Nov 05 00:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_4(8161762) in pi_ohern on r801u32n02 Nov 05 00:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_5(8161763) in pi_ohern on r917u13n01 Nov 05 00:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_6(8161764) in pi_ohern on r917u13n01 Nov 05 00:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_7(8161765) in pi_ohern on r917u13n01 Nov 05 00:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_8(8161766) in pi_ohern on r917u13n02 Nov 05 00:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_9(8161767) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_10(8161768) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_11(8161769) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_12(8161770) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_13(8161771) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_14(8161772) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_15(8161773) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_16(8161774) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_17(8161775) in pi_ohern on r917u13n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_18(8161776) in pi_ohern on r917u13n03 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_19(8161777) in pi_ohern on r917u13n03 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_20(8161778) in pi_ohern on r917u13n03 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_21(8161779) in pi_ohern on r917u13n03 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_22(8161780) in pi_ohern on r917u13n03 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_23(8161781) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_24(8161782) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_25(8161783) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_26(8161784) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_27(8161785) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_28(8161786) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_29(8161787) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_30(8161788) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_31(8161789) in pi_ohern on r917u13n04 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_32(8161790) in pi_ohern on r917u29n02 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_33(8161791) in pi_ohern on r917u29n03 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_34(8161792) in pi_ohern on r917u29n03 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_35(8161793) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_36(8161794) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_37(8161795) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_38(8161796) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_39(8161797) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_40(8161798) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_41(8161799) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_42(8161800) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_43(8161801) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_44(8161802) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_45(8161803) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_46(8161804) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_47(8161805) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_48(8161806) in pi_ohern on r815u03n11 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_49(8161807) in pi_ohern on r815u03n12 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_50(8161808) in pi_ohern on r815u03n12 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_51(8161809) in pi_ohern on r815u03n12 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_52(8161810) in pi_ohern on r815u03n12 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_53(8161811) in pi_ohern on r815u03n12 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_54(8161812) in pi_ohern on r815u03n12 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_55(8161813) in pi_ohern on r913u32n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_56(8161814) in pi_ohern on r913u32n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_57(8161815) in pi_ohern on r913u35n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_58(8161816) in pi_ohern on r903u12n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_59(8161817) in pi_ohern on r903u12n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_60(8161818) in pi_ohern on r903u12n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_61(8161819) in pi_ohern on r903u12n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_62(8161820) in pi_ohern on r903u12n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_63(8161821) in pi_ohern on r903u12n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_64(8161822) in pi_ohern on r903u24n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_65(8161823) in pi_ohern on r903u24n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_66(8161824) in pi_ohern on r903u24n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_67(8161825) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_68(8161826) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_69(8161827) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_70(8161828) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_71(8161829) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_72(8161830) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_73(8161831) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_74(8161832) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_75(8161833) in pi_ohern on r903u26n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159913 in day on r815u35n01 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159914 in day on r815u03n08 Nov 05 00:18:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159915 in day on r815u15n02 Nov 05 00:18:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_41(8161834) in day on r806u09n04 Nov 05 00:18:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3626(8161835) in gpu on r808u14n01 Nov 05 00:18:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3627(8161836) in gpu on r4513u23n01 Nov 05 00:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_45(8159813) WEXITSTATUS 0 Nov 05 00:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_45(8159813) done Nov 05 00:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_1078(8115347) WEXITSTATUS 0 Nov 05 00:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_1078(8115347) done Nov 05 00:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_260(8160208) WEXITSTATUS 0 Nov 05 00:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_260(8160208) done Nov 05 00:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_113(8159881) WEXITSTATUS 0 Nov 05 00:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_113(8159881) done Nov 05 00:18:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_248(8160196) WEXITSTATUS 0 Nov 05 00:18:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_248(8160196) done Nov 05 00:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_108(8159876) WEXITSTATUS 0 Nov 05 00:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_108(8159876) done Nov 05 00:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_25(8135724) WEXITSTATUS 0 Nov 05 00:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_25(8135724) done Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5635(8161837) in scavenge on r801u27n02 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5636(8161838) in scavenge on r802u06n03 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5637(8161839) in scavenge on r903u04n03 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5638(8161840) in scavenge on r911u05n01 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5639(8161841) in scavenge on r805u04n02 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5640(8161842) in scavenge on r806u09n04 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5641(8161843) in scavenge on r918u32n04 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5642(8161844) in scavenge on r816u15n03 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5230(8158633) WEXITSTATUS 1 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5230(8158633) done Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5347(8159025) WEXITSTATUS 1 Nov 05 00:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5347(8159025) done Nov 05 00:19:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161845 InitPrio=61739 usec=4919 Nov 05 00:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_51(8159819) WEXITSTATUS 0 Nov 05 00:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_51(8159819) done Nov 05 00:19:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_257(8160205) WEXITSTATUS 0 Nov 05 00:19:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_257(8160205) done Nov 05 00:19:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_184(8160012) WEXITSTATUS 0 Nov 05 00:19:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_184(8160012) done Nov 05 00:19:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1875(8161846) in scavenge on r815u15n09 Nov 05 00:19:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1876(8161847) in scavenge on r815u29n08 Nov 05 00:19:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _attempt_backfill: 159 pending RPCs at cycle end, consider configuring max_rpc_cnt Nov 05 00:19:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_35(8155794) Nov 05 00:19:13 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5408(8159389) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:19:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_50(8159818) WEXITSTATUS 0 Nov 05 00:19:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_50(8159818) done Nov 05 00:19:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5408(8159389) Nov 05 00:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_173(8160001) WEXITSTATUS 0 Nov 05 00:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_173(8160001) done Nov 05 00:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_187(8160015) WEXITSTATUS 0 Nov 05 00:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_187(8160015) done Nov 05 00:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_177(8160005) WEXITSTATUS 0 Nov 05 00:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_177(8160005) done Nov 05 00:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_258(8160206) WEXITSTATUS 0 Nov 05 00:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_258(8160206) done Nov 05 00:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_17(8159785) WEXITSTATUS 0 Nov 05 00:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_17(8159785) done Nov 05 00:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_269(8160217) WEXITSTATUS 0 Nov 05 00:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_269(8160217) done Nov 05 00:19:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161848 InitPrio=13461 usec=4193 Nov 05 00:19:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161845 NodeList=r806u14n04 #CPUs=1 Partition=day Nov 05 00:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_129(8159957) WEXITSTATUS 0 Nov 05 00:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_129(8159957) done Nov 05 00:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_164(8159992) WEXITSTATUS 0 Nov 05 00:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_164(8159992) done Nov 05 00:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_246(8160194) WEXITSTATUS 0 Nov 05 00:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_246(8160194) done Nov 05 00:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5317(8158845) WEXITSTATUS 1 Nov 05 00:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5317(8158845) done Nov 05 00:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5373(8159266) WEXITSTATUS 1 Nov 05 00:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5373(8159266) done Nov 05 00:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157511 WEXITSTATUS 0 Nov 05 00:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157511 done Nov 05 00:19:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4939(8158148) WEXITSTATUS 1 Nov 05 00:19:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4939(8158148) done Nov 05 00:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_193(8160021) WEXITSTATUS 0 Nov 05 00:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_193(8160021) done Nov 05 00:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_253(8160201) WEXITSTATUS 0 Nov 05 00:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_253(8160201) done Nov 05 00:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157516 WEXITSTATUS 0 Nov 05 00:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157516 done Nov 05 00:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_93(8159861) WEXITSTATUS 0 Nov 05 00:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_93(8159861) done Nov 05 00:19:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_110(8159878) WEXITSTATUS 0 Nov 05 00:19:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_110(8159878) done Nov 05 00:19:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_275(8160223) WEXITSTATUS 0 Nov 05 00:19:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_275(8160223) done Nov 05 00:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_165(8159993) WEXITSTATUS 0 Nov 05 00:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_165(8159993) done Nov 05 00:19:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161849 InitPrio=59533 usec=4010 Nov 05 00:19:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161850 InitPrio=59533 usec=3107 Nov 05 00:19:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161851 InitPrio=59533 usec=2943 Nov 05 00:19:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161852 InitPrio=59533 usec=2682 Nov 05 00:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_274(8160222) WEXITSTATUS 0 Nov 05 00:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_274(8160222) done Nov 05 00:19:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161849 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:19:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161850 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:19:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161851 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:19:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8161852 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5341(8159019) WEXITSTATUS 1 Nov 05 00:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5341(8159019) done Nov 05 00:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1468(8138363) WEXITSTATUS 0 Nov 05 00:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1468(8138363) done Nov 05 00:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_89(8159857) WEXITSTATUS 0 Nov 05 00:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_89(8159857) done Nov 05 00:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_181(8160009) WEXITSTATUS 0 Nov 05 00:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_181(8160009) done Nov 05 00:19:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_249(8160197) WEXITSTATUS 0 Nov 05 00:19:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_249(8160197) done Nov 05 00:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_41(8158711) WEXITSTATUS 0 Nov 05 00:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_41(8158711) done Nov 05 00:19:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_85(8159853) WEXITSTATUS 0 Nov 05 00:19:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_85(8159853) done Nov 05 00:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_112(8159880) WEXITSTATUS 0 Nov 05 00:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_112(8159880) done Nov 05 00:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_145(8159973) WEXITSTATUS 0 Nov 05 00:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_145(8159973) done Nov 05 00:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_27(8159795) WEXITSTATUS 0 Nov 05 00:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_27(8159795) done Nov 05 00:20:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2039(8151599) WEXITSTATUS 0 Nov 05 00:20:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2039(8151599) done Nov 05 00:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_191(8160019) WEXITSTATUS 0 Nov 05 00:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_191(8160019) done Nov 05 00:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_156(8159984) WEXITSTATUS 0 Nov 05 00:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_156(8159984) done Nov 05 00:20:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_303(8160374) WEXITSTATUS 0 Nov 05 00:20:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_303(8160374) done Nov 05 00:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_142(8159970) WEXITSTATUS 0 Nov 05 00:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_142(8159970) done Nov 05 00:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_150(8159978) WEXITSTATUS 0 Nov 05 00:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_150(8159978) done Nov 05 00:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_174(8160002) WEXITSTATUS 0 Nov 05 00:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_174(8160002) done Nov 05 00:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_160(8159988) WEXITSTATUS 0 Nov 05 00:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_160(8159988) done Nov 05 00:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157514 WEXITSTATUS 0 Nov 05 00:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157514 done Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8152914 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_175(8160003) WEXITSTATUS 0 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_175(8160003) done Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:20:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:20:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5609(8161696) WEXITSTATUS 1 Nov 05 00:20:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5609(8161696) done Nov 05 00:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161849 WEXITSTATUS 0 Nov 05 00:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161849 done Nov 05 00:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161850 WEXITSTATUS 0 Nov 05 00:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161850 done Nov 05 00:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_18(8159786) WEXITSTATUS 0 Nov 05 00:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_18(8159786) done Nov 05 00:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161851 WEXITSTATUS 0 Nov 05 00:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161851 done Nov 05 00:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161852 WEXITSTATUS 0 Nov 05 00:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161852 done Nov 05 00:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_58(8159826) WEXITSTATUS 0 Nov 05 00:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_58(8159826) done Nov 05 00:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_28(8159796) WEXITSTATUS 0 Nov 05 00:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_28(8159796) done Nov 05 00:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_5(8159748) WEXITSTATUS 0 Nov 05 00:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_5(8159748) done Nov 05 00:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2198(8151990) WEXITSTATUS 0 Nov 05 00:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2198(8151990) done Nov 05 00:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_105(8159873) WEXITSTATUS 0 Nov 05 00:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_105(8159873) done Nov 05 00:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_30(8159798) WEXITSTATUS 0 Nov 05 00:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_30(8159798) done Nov 05 00:20:40 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:40 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:41 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:42 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:43 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:44 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:44 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:46 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:48 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8161853 InitPrio=21238 usec=4256 Nov 05 00:20:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:50 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_245(8160193) WEXITSTATUS 0 Nov 05 00:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_245(8160193) done Nov 05 00:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_59(8159827) WEXITSTATUS 0 Nov 05 00:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_59(8159827) done Nov 05 00:20:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_37(8159805) WEXITSTATUS 0 Nov 05 00:20:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_37(8159805) done Nov 05 00:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5157(8158504) WEXITSTATUS 1 Nov 05 00:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5157(8158504) done Nov 05 00:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_148(8159976) WEXITSTATUS 0 Nov 05 00:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_148(8159976) done Nov 05 00:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_169(8159997) WEXITSTATUS 0 Nov 05 00:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_169(8159997) done Nov 05 00:21:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_32(8159800) WEXITSTATUS 0 Nov 05 00:21:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_32(8159800) done Nov 05 00:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157513 WEXITSTATUS 0 Nov 05 00:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157513 done Nov 05 00:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_146(8159974) WEXITSTATUS 0 Nov 05 00:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_146(8159974) done Nov 05 00:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_302(8160373) WEXITSTATUS 0 Nov 05 00:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_302(8160373) done Nov 05 00:21:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_159(8159987) WEXITSTATUS 0 Nov 05 00:21:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_159(8159987) done Nov 05 00:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_149(8159977) WEXITSTATUS 0 Nov 05 00:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_149(8159977) done Nov 05 00:21:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_304(8160375) WEXITSTATUS 0 Nov 05 00:21:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_304(8160375) done Nov 05 00:21:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_106(8159874) WEXITSTATUS 0 Nov 05 00:21:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_106(8159874) done Nov 05 00:21:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_54(8159822) WEXITSTATUS 0 Nov 05 00:21:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_54(8159822) done Nov 05 00:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_47(8158726) WEXITSTATUS 0 Nov 05 00:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_47(8158726) done Nov 05 00:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_53(8159821) WEXITSTATUS 0 Nov 05 00:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_53(8159821) done Nov 05 00:21:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_377(8160579) WEXITSTATUS 0 Nov 05 00:21:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_377(8160579) done Nov 05 00:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_155(8159983) WEXITSTATUS 0 Nov 05 00:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_155(8159983) done Nov 05 00:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_24(8159792) WEXITSTATUS 0 Nov 05 00:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_24(8159792) done Nov 05 00:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_86(8159854) WEXITSTATUS 0 Nov 05 00:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_86(8159854) done Nov 05 00:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_66(8159834) WEXITSTATUS 0 Nov 05 00:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_66(8159834) done Nov 05 00:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_100(8159868) WEXITSTATUS 0 Nov 05 00:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_100(8159868) done Nov 05 00:21:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_179(8160007) WEXITSTATUS 0 Nov 05 00:21:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_179(8160007) done Nov 05 00:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_26(8159794) WEXITSTATUS 0 Nov 05 00:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_26(8159794) done Nov 05 00:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1474(8138475) WEXITSTATUS 0 Nov 05 00:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1474(8138475) done Nov 05 00:21:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_62(8159830) WEXITSTATUS 0 Nov 05 00:21:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_62(8159830) done Nov 05 00:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5615(8161716) WEXITSTATUS 1 Nov 05 00:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5615(8161716) done Nov 05 00:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_163(8159991) WEXITSTATUS 0 Nov 05 00:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_163(8159991) done Nov 05 00:21:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_16(8159784) WEXITSTATUS 0 Nov 05 00:21:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_16(8159784) done Nov 05 00:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_64(8159832) WEXITSTATUS 0 Nov 05 00:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_64(8159832) done Nov 05 00:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_171(8159999) WEXITSTATUS 0 Nov 05 00:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_171(8159999) done Nov 05 00:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5621(8161722) WEXITSTATUS 1 Nov 05 00:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5621(8161722) done Nov 05 00:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_74(8159842) WEXITSTATUS 0 Nov 05 00:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_74(8159842) done Nov 05 00:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156078 WEXITSTATUS 0 Nov 05 00:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8156078 done Nov 05 00:21:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_117(8159945) WEXITSTATUS 0 Nov 05 00:21:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_117(8159945) done Nov 05 00:21:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_15(8159783) WEXITSTATUS 0 Nov 05 00:21:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_15(8159783) done Nov 05 00:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_43(8159811) WEXITSTATUS 0 Nov 05 00:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_43(8159811) done Nov 05 00:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_261(8160209) WEXITSTATUS 0 Nov 05 00:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_261(8160209) done Nov 05 00:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_144(8159972) WEXITSTATUS 0 Nov 05 00:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_144(8159972) done Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161848 in pi_balou on r802u30n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_76(8161854) in pi_ohern on r801u32n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_77(8161855) in pi_ohern on r801u32n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_78(8161856) in pi_ohern on r801u32n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_79(8161857) in pi_ohern on r801u32n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_80(8161858) in pi_ohern on r801u32n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_81(8161859) in pi_ohern on r917u13n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_82(8161860) in pi_ohern on r917u13n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_83(8161861) in pi_ohern on r917u13n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_84(8161862) in pi_ohern on r917u13n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_85(8161863) in pi_ohern on r917u13n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_86(8161864) in pi_ohern on r917u13n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_87(8161865) in pi_ohern on r917u13n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_88(8161866) in pi_ohern on r917u13n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_89(8161867) in pi_ohern on r917u13n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_90(8161868) in pi_ohern on r917u13n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_91(8161869) in pi_ohern on r917u13n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_92(8161870) in pi_ohern on r917u13n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_93(8161871) in pi_ohern on r917u13n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_94(8161872) in pi_ohern on r917u13n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_95(8161873) in pi_ohern on r917u13n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_96(8161874) in pi_ohern on r917u13n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_97(8161875) in pi_ohern on r917u13n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_98(8161876) in pi_ohern on r917u13n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_99(8161877) in pi_ohern on r917u13n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_100(8161878) in pi_ohern on r917u13n04 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_101(8161879) in pi_ohern on r917u29n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_102(8161880) in pi_ohern on r917u29n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_103(8161881) in pi_ohern on r917u29n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_104(8161882) in pi_ohern on r917u29n02 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_105(8161883) in pi_ohern on r917u29n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_106(8161884) in pi_ohern on r917u29n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_107(8161885) in pi_ohern on r917u29n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_108(8161886) in pi_ohern on r917u29n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_109(8161887) in pi_ohern on r917u29n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_110(8161888) in pi_ohern on r917u29n04 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_111(8161889) in pi_ohern on r815u03n11 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_112(8161890) in pi_ohern on r815u03n11 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_113(8161891) in pi_ohern on r815u03n11 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_114(8161892) in pi_ohern on r815u03n11 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_115(8161893) in pi_ohern on r815u03n11 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_116(8161894) in pi_ohern on r815u03n11 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_117(8161895) in pi_ohern on r815u03n11 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_118(8161896) in pi_ohern on r815u03n12 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_119(8161897) in pi_ohern on r815u03n12 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_120(8161898) in pi_ohern on r815u03n12 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_121(8161899) in pi_ohern on r815u03n12 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_122(8161900) in pi_ohern on r815u03n12 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_123(8161901) in pi_ohern on r815u03n12 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_124(8161902) in pi_ohern on r911u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_125(8161903) in pi_ohern on r911u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_126(8161904) in pi_ohern on r911u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_127(8161905) in pi_ohern on r911u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_128(8161906) in pi_ohern on r911u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_129(8161907) in pi_ohern on r911u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_130(8161908) in pi_ohern on r911u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_131(8161909) in pi_ohern on r807u22n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_132(8161910) in pi_ohern on r807u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_133(8161911) in pi_ohern on r807u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_134(8161912) in pi_ohern on r913u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_135(8161913) in pi_ohern on r913u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_136(8161914) in pi_ohern on r913u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_137(8161915) in pi_ohern on r913u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_138(8161916) in pi_ohern on r913u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_139(8161917) in pi_ohern on r913u32n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_140(8161918) in pi_ohern on r913u35n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_141(8161919) in pi_ohern on r913u35n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_142(8161920) in pi_ohern on r903u12n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_143(8161921) in pi_ohern on r903u12n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_144(8161922) in pi_ohern on r903u12n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_145(8161923) in pi_ohern on r903u12n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_146(8161924) in pi_ohern on r903u12n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_147(8161925) in pi_ohern on r903u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_148(8161926) in pi_ohern on r903u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_149(8161927) in pi_ohern on r903u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_150(8161928) in pi_ohern on r903u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_151(8161929) in pi_ohern on r903u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_152(8161930) in pi_ohern on r903u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_153(8161931) in pi_ohern on r903u24n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_154(8161932) in pi_ohern on r903u26n01 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159905 in day on r909u01n04 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159910 in day on r909u07n03 Nov 05 00:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159918 in day on r815u03n03 Nov 05 00:21:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161853 in day on r909u29n03,r918u09n04 Nov 05 00:21:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_42(8161933) in day on r807u08n01 Nov 05 00:21:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_43(8161934) in day on r805u02n02 Nov 05 00:21:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_44(8161935) in day on r805u25n01 Nov 05 00:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_241(8160189) WEXITSTATUS 0 Nov 05 00:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_241(8160189) done Nov 05 00:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_305(8160376) WEXITSTATUS 0 Nov 05 00:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_305(8160376) done Nov 05 00:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_92(8159860) WEXITSTATUS 0 Nov 05 00:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_92(8159860) done Nov 05 00:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_35(8159803) WEXITSTATUS 0 Nov 05 00:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_35(8159803) done Nov 05 00:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_46(8158725) WEXITSTATUS 0 Nov 05 00:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_46(8158725) done Nov 05 00:21:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5643(8161936) in scavenge on r801u27n02 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5644(8161937) in scavenge on r802u11n04 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5645(8161938) in scavenge on r903u04n01 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5646(8161939) in scavenge on r815u29n12 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5647(8161940) in scavenge on r816u03n10 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5648(8161941) in scavenge on r816u23n11 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5649(8161942) in scavenge on r816u35n12 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5650(8161943) in scavenge on r904u04n03 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5651(8161944) in scavenge on r913u24n04 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5652(8161945) in scavenge on r815u23n06 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5653(8161946) in scavenge on r816u23n06 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5654(8161947) in scavenge on r816u29n02 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5655(8161948) in scavenge on r908u24n02 Nov 05 00:21:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5656(8161949) in scavenge on r904u13n01 Nov 05 00:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_170(8159998) WEXITSTATUS 0 Nov 05 00:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_170(8159998) done Nov 05 00:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1276(8135231) WEXITSTATUS 0 Nov 05 00:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1276(8135231) done Nov 05 00:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_21(8159789) WEXITSTATUS 0 Nov 05 00:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_21(8159789) done Nov 05 00:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_71(8159839) WEXITSTATUS 0 Nov 05 00:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_71(8159839) done Nov 05 00:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_88(8159856) WEXITSTATUS 0 Nov 05 00:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_88(8159856) done Nov 05 00:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_254(8160202) WEXITSTATUS 0 Nov 05 00:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_254(8160202) done Nov 05 00:22:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8137580 WEXITSTATUS 0 Nov 05 00:22:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8137580 done Nov 05 00:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_379(8160581) WEXITSTATUS 0 Nov 05 00:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_379(8160581) done Nov 05 00:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5652(8161945) WEXITSTATUS 1 Nov 05 00:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5652(8161945) done Nov 05 00:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_375(8160577) WEXITSTATUS 0 Nov 05 00:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_375(8160577) done Nov 05 00:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_250(8160198) WEXITSTATUS 0 Nov 05 00:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_250(8160198) done Nov 05 00:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_67(8159835) WEXITSTATUS 0 Nov 05 00:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_67(8159835) done Nov 05 00:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_161(8159989) WEXITSTATUS 0 Nov 05 00:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_161(8159989) done Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_120(8159948) WEXITSTATUS 0 Nov 05 00:22:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_120(8159948) done Nov 05 00:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_153(8159981) WEXITSTATUS 0 Nov 05 00:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_153(8159981) done Nov 05 00:22:18 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3945(8145777) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:22:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8153134 Nov 05 00:22:18 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4357(8154967) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:22:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_36(8155847) Nov 05 00:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_229(8160177) WEXITSTATUS 0 Nov 05 00:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_229(8160177) done Nov 05 00:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_69(8159837) WEXITSTATUS 0 Nov 05 00:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_69(8159837) done Nov 05 00:22:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3945(8145777) Nov 05 00:22:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4357(8154967) Nov 05 00:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161853 WEXITSTATUS 1 Nov 05 00:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161853 done Nov 05 00:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_143(8159971) WEXITSTATUS 0 Nov 05 00:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_143(8159971) done Nov 05 00:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_264(8160212) WEXITSTATUS 0 Nov 05 00:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_264(8160212) done Nov 05 00:22:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_266(8160214) WEXITSTATUS 0 Nov 05 00:22:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_266(8160214) done Nov 05 00:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5406(8159387) WEXITSTATUS 1 Nov 05 00:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5406(8159387) done Nov 05 00:22:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_189(8160017) WEXITSTATUS 0 Nov 05 00:22:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_189(8160017) done Nov 05 00:22:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_104(8159872) WEXITSTATUS 0 Nov 05 00:22:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_104(8159872) done Nov 05 00:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_61(8159829) WEXITSTATUS 0 Nov 05 00:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_61(8159829) done Nov 05 00:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_121(8159949) WEXITSTATUS 0 Nov 05 00:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_121(8159949) done Nov 05 00:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_23(8159791) WEXITSTATUS 0 Nov 05 00:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_23(8159791) done Nov 05 00:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_374(8160576) WEXITSTATUS 0 Nov 05 00:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_374(8160576) done Nov 05 00:23:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_180(8160008) WEXITSTATUS 0 Nov 05 00:23:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_180(8160008) done Nov 05 00:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4667(8156329) WEXITSTATUS 1 Nov 05 00:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4667(8156329) done Nov 05 00:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_172(8160000) WEXITSTATUS 0 Nov 05 00:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_172(8160000) done Nov 05 00:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_190(8160018) WEXITSTATUS 0 Nov 05 00:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_190(8160018) done Nov 05 00:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_182(8160010) WEXITSTATUS 0 Nov 05 00:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_182(8160010) done Nov 05 00:23:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_255(8160203) WEXITSTATUS 0 Nov 05 00:23:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_255(8160203) done Nov 05 00:23:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_101(8159869) WEXITSTATUS 0 Nov 05 00:23:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_101(8159869) done Nov 05 00:23:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_205(8160153) WEXITSTATUS 0 Nov 05 00:23:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_205(8160153) done Nov 05 00:23:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_99(8159867) WEXITSTATUS 0 Nov 05 00:23:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_99(8159867) done Nov 05 00:23:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157512 WEXITSTATUS 0 Nov 05 00:23:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157512 done Nov 05 00:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_231(8160179) WEXITSTATUS 0 Nov 05 00:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_231(8160179) done Nov 05 00:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_384(8160586) WEXITSTATUS 0 Nov 05 00:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_384(8160586) done Nov 05 00:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161728 WEXITSTATUS 0 Nov 05 00:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161728 done Nov 05 00:23:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:23:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:23:20 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:23:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_151(8159979) WEXITSTATUS 0 Nov 05 00:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_151(8159979) done Nov 05 00:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_34(8159802) WEXITSTATUS 0 Nov 05 00:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_34(8159802) done Nov 05 00:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_72(8159840) WEXITSTATUS 0 Nov 05 00:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_72(8159840) done Nov 05 00:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_40(8159808) WEXITSTATUS 0 Nov 05 00:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_40(8159808) done Nov 05 00:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_263(8160211) WEXITSTATUS 0 Nov 05 00:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_263(8160211) done Nov 05 00:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_373(8160575) WEXITSTATUS 0 Nov 05 00:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_373(8160575) done Nov 05 00:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_247(8160195) WEXITSTATUS 0 Nov 05 00:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_247(8160195) done Nov 05 00:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_240(8160188) WEXITSTATUS 0 Nov 05 00:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_240(8160188) done Nov 05 00:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_219(8160167) WEXITSTATUS 0 Nov 05 00:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_219(8160167) done Nov 05 00:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5520(8160699) WEXITSTATUS 1 Nov 05 00:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5520(8160699) done Nov 05 00:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_75(8159843) WEXITSTATUS 0 Nov 05 00:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_75(8159843) done Nov 05 00:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_109(8159877) WEXITSTATUS 0 Nov 05 00:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_109(8159877) done Nov 05 00:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_317(8160388) WEXITSTATUS 0 Nov 05 00:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_317(8160388) done Nov 05 00:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_376(8160578) WEXITSTATUS 0 Nov 05 00:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_376(8160578) done Nov 05 00:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_70(8159838) WEXITSTATUS 0 Nov 05 00:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_70(8159838) done Nov 05 00:23:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_208(8160156) WEXITSTATUS 0 Nov 05 00:23:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_208(8160156) done Nov 05 00:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_206(8160154) WEXITSTATUS 0 Nov 05 00:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_206(8160154) done Nov 05 00:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_277(8160225) WEXITSTATUS 0 Nov 05 00:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_277(8160225) done Nov 05 00:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4580(8156093) WEXITSTATUS 1 Nov 05 00:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4580(8156093) done Nov 05 00:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_118(8159946) WEXITSTATUS 0 Nov 05 00:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_118(8159946) done Nov 05 00:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_271(8160219) WEXITSTATUS 0 Nov 05 00:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_271(8160219) done Nov 05 00:23:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_290(8160361) WEXITSTATUS 0 Nov 05 00:23:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_290(8160361) done Nov 05 00:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154599 WEXITSTATUS 0 Nov 05 00:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154599 done Nov 05 00:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5656(8161949) WEXITSTATUS 1 Nov 05 00:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5656(8161949) done Nov 05 00:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5143(8158490) WEXITSTATUS 1 Nov 05 00:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5143(8158490) done Nov 05 00:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_381(8160583) WEXITSTATUS 0 Nov 05 00:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_381(8160583) done Nov 05 00:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5407(8159388) WEXITSTATUS 1 Nov 05 00:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5407(8159388) done Nov 05 00:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_330(8160401) WEXITSTATUS 0 Nov 05 00:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_330(8160401) done Nov 05 00:23:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_209(8160157) WEXITSTATUS 0 Nov 05 00:23:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_209(8160157) done Nov 05 00:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_203(8160151) WEXITSTATUS 0 Nov 05 00:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_203(8160151) done Nov 05 00:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_210(8160158) WEXITSTATUS 0 Nov 05 00:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_210(8160158) done Nov 05 00:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5195(8158558) WEXITSTATUS 1 Nov 05 00:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5195(8158558) done Nov 05 00:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_102(8159870) WEXITSTATUS 0 Nov 05 00:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_102(8159870) done Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:24:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_273(8160221) WEXITSTATUS 0 Nov 05 00:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_273(8160221) done Nov 05 00:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_207(8160155) WEXITSTATUS 0 Nov 05 00:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_207(8160155) done Nov 05 00:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_265(8160213) WEXITSTATUS 0 Nov 05 00:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_265(8160213) done Nov 05 00:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_186(8160014) WEXITSTATUS 0 Nov 05 00:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_186(8160014) done Nov 05 00:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_321(8160392) WEXITSTATUS 0 Nov 05 00:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_321(8160392) done Nov 05 00:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_268(8160216) WEXITSTATUS 0 Nov 05 00:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_268(8160216) done Nov 05 00:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_60(8159828) WEXITSTATUS 0 Nov 05 00:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_60(8159828) done Nov 05 00:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_103(8159871) WEXITSTATUS 0 Nov 05 00:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_103(8159871) done Nov 05 00:24:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_213(8160161) WEXITSTATUS 0 Nov 05 00:24:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_213(8160161) done Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_155(8161950) in pi_ohern on r801u32n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_156(8161951) in pi_ohern on r801u32n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_157(8161952) in pi_ohern on r801u32n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_158(8161953) in pi_ohern on r801u32n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_159(8161954) in pi_ohern on r917u13n01 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_160(8161955) in pi_ohern on r917u13n01 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_161(8161956) in pi_ohern on r917u13n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_162(8161957) in pi_ohern on r917u13n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_163(8161958) in pi_ohern on r917u13n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_164(8161959) in pi_ohern on r917u13n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_165(8161960) in pi_ohern on r917u13n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_166(8161961) in pi_ohern on r917u13n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_167(8161962) in pi_ohern on r917u13n02 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_31(8159799) WEXITSTATUS 0 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_31(8159799) done Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_201(8160149) WEXITSTATUS 0 Nov 05 00:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_201(8160149) done Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_168(8161963) in pi_ohern on r917u13n02 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_169(8161964) in pi_ohern on r917u13n02 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_170(8161965) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_171(8161966) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_172(8161967) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_173(8161968) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_174(8161969) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_175(8161970) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_176(8161971) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_177(8161972) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_178(8161973) in pi_ohern on r917u13n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_179(8161974) in pi_ohern on r917u13n04 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_180(8161975) in pi_ohern on r917u13n04 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_181(8161976) in pi_ohern on r917u13n04 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_182(8161977) in pi_ohern on r917u29n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_183(8161978) in pi_ohern on r917u29n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_184(8161979) in pi_ohern on r917u29n02 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_185(8161980) in pi_ohern on r917u29n02 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_186(8161981) in pi_ohern on r917u29n02 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_187(8161982) in pi_ohern on r917u29n02 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_188(8161983) in pi_ohern on r917u29n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_189(8161984) in pi_ohern on r917u29n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_190(8161985) in pi_ohern on r917u29n03 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_191(8161986) in pi_ohern on r917u29n04 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_192(8161987) in pi_ohern on r917u29n04 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_193(8161988) in pi_ohern on r815u03n11 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_194(8161989) in pi_ohern on r815u03n11 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_195(8161990) in pi_ohern on r815u03n11 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_196(8161991) in pi_ohern on r815u03n11 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_197(8161992) in pi_ohern on r815u03n11 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_198(8161993) in pi_ohern on r815u03n12 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_199(8161994) in pi_ohern on r815u03n12 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_200(8161995) in pi_ohern on r815u03n12 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_201(8161996) in pi_ohern on r815u03n12 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_202(8161997) in pi_ohern on r815u03n12 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_203(8161998) in pi_ohern on r815u03n12 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_204(8161999) in pi_ohern on r815u03n12 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_205(8162000) in pi_ohern on r911u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_206(8162001) in pi_ohern on r911u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_207(8162002) in pi_ohern on r911u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_208(8162003) in pi_ohern on r911u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_209(8162004) in pi_ohern on r911u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_210(8162005) in pi_ohern on r911u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_211(8162006) in pi_ohern on r911u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_212(8162007) in pi_ohern on r807u22n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_213(8162008) in pi_ohern on r807u24n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_214(8162009) in pi_ohern on r913u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_215(8162010) in pi_ohern on r913u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_216(8162011) in pi_ohern on r913u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_217(8162012) in pi_ohern on r913u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_218(8162013) in pi_ohern on r913u32n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_219(8162014) in pi_ohern on r913u35n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_220(8162015) in pi_ohern on r913u35n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_221(8162016) in pi_ohern on r903u12n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_222(8162017) in pi_ohern on r903u24n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_223(8162018) in pi_ohern on r903u24n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_224(8162019) in pi_ohern on r903u24n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_225(8162020) in pi_ohern on r903u24n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_226(8162021) in pi_ohern on r903u26n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159916 in day on r918u09n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159917 in day on r918u09n01 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159921 in day on r815u09n02 Nov 05 00:24:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159922 in day on r816u15n07 Nov 05 00:24:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_45(8162022) in day on r805u11n02 Nov 05 00:24:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_46(8162023) in day on r807u05n03 Nov 05 00:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_262(8160210) WEXITSTATUS 0 Nov 05 00:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_262(8160210) done Nov 05 00:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_162(8159990) WEXITSTATUS 0 Nov 05 00:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_162(8159990) done Nov 05 00:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_192(8160020) WEXITSTATUS 0 Nov 05 00:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_192(8160020) done Nov 05 00:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_378(8160580) WEXITSTATUS 0 Nov 05 00:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_378(8160580) done Nov 05 00:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5397(8159335) WEXITSTATUS 1 Nov 05 00:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5397(8159335) done Nov 05 00:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5657(8162024) in scavenge on r816u15n10 Nov 05 00:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5658(8162025) in scavenge on r816u35n12 Nov 05 00:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5659(8162026) in scavenge on r806u02n03 Nov 05 00:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5660(8162027) in scavenge on r918u09n01 Nov 05 00:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5661(8162028) in scavenge on r805u09n01 Nov 05 00:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5662(8162029) in scavenge on r805u23n02 Nov 05 00:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5663(8162030) in scavenge on r815u23n06 Nov 05 00:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5664(8162031) in scavenge on r904u13n01 Nov 05 00:24:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2029(8125796) Nov 05 00:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_19(8159787) WEXITSTATUS 0 Nov 05 00:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_19(8159787) done Nov 05 00:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_380(8160582) WEXITSTATUS 0 Nov 05 00:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_380(8160582) done Nov 05 00:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_331(8160402) WEXITSTATUS 0 Nov 05 00:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_331(8160402) done Nov 05 00:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_147(8159975) WEXITSTATUS 0 Nov 05 00:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_147(8159975) done Nov 05 00:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157518 WEXITSTATUS 0 Nov 05 00:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157518 done Nov 05 00:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_230(8160178) WEXITSTATUS 0 Nov 05 00:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_230(8160178) done Nov 05 00:25:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_212(8160160) WEXITSTATUS 0 Nov 05 00:25:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_212(8160160) done Nov 05 00:25:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5463(8159779) WEXITSTATUS 1 Nov 05 00:25:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5463(8159779) done Nov 05 00:25:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_233(8160181) WEXITSTATUS 0 Nov 05 00:25:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_233(8160181) done Nov 05 00:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_87(8159855) WEXITSTATUS 0 Nov 05 00:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_87(8159855) done Nov 05 00:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_122(8159950) WEXITSTATUS 0 Nov 05 00:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_122(8159950) done Nov 05 00:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_204(8160152) WEXITSTATUS 0 Nov 05 00:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_204(8160152) done Nov 05 00:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_315(8160386) WEXITSTATUS 0 Nov 05 00:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_315(8160386) done Nov 05 00:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_188(8160016) WEXITSTATUS 0 Nov 05 00:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_188(8160016) done Nov 05 00:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5415(8159460) WEXITSTATUS 1 Nov 05 00:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5415(8159460) done Nov 05 00:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_95(8159863) WEXITSTATUS 0 Nov 05 00:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_95(8159863) done Nov 05 00:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_96(8159864) WEXITSTATUS 0 Nov 05 00:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_96(8159864) done Nov 05 00:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_41(8159809) WEXITSTATUS 0 Nov 05 00:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_41(8159809) done Nov 05 00:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_323(8160394) WEXITSTATUS 0 Nov 05 00:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_323(8160394) done Nov 05 00:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_211(8160159) WEXITSTATUS 0 Nov 05 00:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_211(8160159) done Nov 05 00:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_328(8160399) WEXITSTATUS 0 Nov 05 00:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_328(8160399) done Nov 05 00:25:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4688(8156350) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_154(8159982) WEXITSTATUS 0 Nov 05 00:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_154(8159982) done Nov 05 00:25:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_202(8160150) WEXITSTATUS 0 Nov 05 00:25:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_202(8160150) done Nov 05 00:25:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4688(8156350) Nov 05 00:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157517 WEXITSTATUS 0 Nov 05 00:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157517 done Nov 05 00:25:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_124(8159952) WEXITSTATUS 0 Nov 05 00:25:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_124(8159952) done Nov 05 00:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_324(8160395) WEXITSTATUS 0 Nov 05 00:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_324(8160395) done Nov 05 00:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_333(8160404) WEXITSTATUS 0 Nov 05 00:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_333(8160404) done Nov 05 00:25:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_152(8159980) WEXITSTATUS 0 Nov 05 00:25:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_152(8159980) done Nov 05 00:25:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2354(8152398) WEXITSTATUS 0 Nov 05 00:25:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2354(8152398) done Nov 05 00:25:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_310(8160381) WEXITSTATUS 0 Nov 05 00:25:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_310(8160381) done Nov 05 00:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5389(8159327) WEXITSTATUS 1 Nov 05 00:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5389(8159327) done Nov 05 00:25:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_98(8159866) WEXITSTATUS 0 Nov 05 00:25:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_98(8159866) done Nov 05 00:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_319(8160390) WEXITSTATUS 0 Nov 05 00:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_319(8160390) done Nov 05 00:25:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1100(8131592) WEXITSTATUS 0 Nov 05 00:25:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1100(8131592) done Nov 05 00:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_224(8160172) WEXITSTATUS 0 Nov 05 00:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_224(8160172) done Nov 05 00:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_97(8159865) WEXITSTATUS 0 Nov 05 00:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_97(8159865) done Nov 05 00:25:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_235(8160183) WEXITSTATUS 0 Nov 05 00:25:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_235(8160183) done Nov 05 00:25:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_37(8155877) Nov 05 00:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_312(8160383) WEXITSTATUS 0 Nov 05 00:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_312(8160383) done Nov 05 00:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_119(8159947) WEXITSTATUS 0 Nov 05 00:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_119(8159947) done Nov 05 00:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_123(8159951) WEXITSTATUS 0 Nov 05 00:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_123(8159951) done Nov 05 00:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_434(8160752) WEXITSTATUS 0 Nov 05 00:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_434(8160752) done Nov 05 00:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154611 WEXITSTATUS 0 Nov 05 00:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154611 done Nov 05 00:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157519 WEXITSTATUS 0 Nov 05 00:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157519 done Nov 05 00:26:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_308(8160379) WEXITSTATUS 0 Nov 05 00:26:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_308(8160379) done Nov 05 00:26:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_242(8160190) WEXITSTATUS 0 Nov 05 00:26:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_242(8160190) done Nov 05 00:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_116(8159944) WEXITSTATUS 0 Nov 05 00:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_116(8159944) done Nov 05 00:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_236(8160184) WEXITSTATUS 0 Nov 05 00:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_236(8160184) done Nov 05 00:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5607(8161694) WEXITSTATUS 1 Nov 05 00:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5607(8161694) done Nov 05 00:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_313(8160384) WEXITSTATUS 0 Nov 05 00:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_313(8160384) done Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_307(8160378) WEXITSTATUS 0 Nov 05 00:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_307(8160378) done Nov 05 00:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_114(8159942) WEXITSTATUS 0 Nov 05 00:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_114(8159942) done Nov 05 00:26:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_0(8155889) Nov 05 00:26:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8153363 Nov 05 00:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_429(8160747) WEXITSTATUS 0 Nov 05 00:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_429(8160747) done Nov 05 00:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_115(8159943) WEXITSTATUS 0 Nov 05 00:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_115(8159943) done Nov 05 00:26:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144133 NodeList=r816u09n12,r917u01n01 #CPUs=96 Partition=scavenge Nov 05 00:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154602 WEXITSTATUS 0 Nov 05 00:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154602 done Nov 05 00:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_329(8160400) WEXITSTATUS 0 Nov 05 00:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_329(8160400) done Nov 05 00:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5060(8158365) WEXITSTATUS 1 Nov 05 00:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5060(8158365) done Nov 05 00:26:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_325(8160396) WEXITSTATUS 0 Nov 05 00:26:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_325(8160396) done Nov 05 00:26:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_218(8160166) WEXITSTATUS 0 Nov 05 00:26:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_218(8160166) done Nov 05 00:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_387(8160589) WEXITSTATUS 0 Nov 05 00:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_387(8160589) done Nov 05 00:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3616(8161397) WEXITSTATUS 0 Nov 05 00:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3616(8161397) done Nov 05 00:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_232(8160180) WEXITSTATUS 0 Nov 05 00:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_232(8160180) done Nov 05 00:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_383(8160585) WEXITSTATUS 0 Nov 05 00:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_383(8160585) done Nov 05 00:26:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_306(8160377) WEXITSTATUS 0 Nov 05 00:26:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_306(8160377) done Nov 05 00:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_73(8159841) WEXITSTATUS 0 Nov 05 00:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_73(8159841) done Nov 05 00:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5483(8160132) WEXITSTATUS 1 Nov 05 00:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5483(8160132) done Nov 05 00:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_228(8160176) WEXITSTATUS 0 Nov 05 00:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_228(8160176) done Nov 05 00:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_267(8160215) WEXITSTATUS 0 Nov 05 00:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_267(8160215) done Nov 05 00:26:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_432(8160750) WEXITSTATUS 0 Nov 05 00:26:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_432(8160750) done Nov 05 00:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_234(8160182) WEXITSTATUS 0 Nov 05 00:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_234(8160182) done Nov 05 00:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2727(8128159) WEXITSTATUS 1 Nov 05 00:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2727(8128159) done Nov 05 00:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_198(8160146) WEXITSTATUS 0 Nov 05 00:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_198(8160146) done Nov 05 00:27:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_311(8160382) WEXITSTATUS 0 Nov 05 00:27:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_311(8160382) done Nov 05 00:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_318(8160389) WEXITSTATUS 0 Nov 05 00:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_318(8160389) done Nov 05 00:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5419(8159464) WEXITSTATUS 1 Nov 05 00:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5419(8159464) done Nov 05 00:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160708 WEXITSTATUS 0 Nov 05 00:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160708 done Nov 05 00:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_309(8160380) WEXITSTATUS 0 Nov 05 00:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_309(8160380) done Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_227(8162032) in pi_ohern on r801u32n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_228(8162033) in pi_ohern on r801u32n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_229(8162034) in pi_ohern on r801u32n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_230(8162035) in pi_ohern on r801u32n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_231(8162036) in pi_ohern on r801u32n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_232(8162037) in pi_ohern on r801u32n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_233(8162038) in pi_ohern on r801u32n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_234(8162039) in pi_ohern on r801u32n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_235(8162040) in pi_ohern on r917u13n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_236(8162041) in pi_ohern on r917u13n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_237(8162042) in pi_ohern on r917u13n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_238(8162043) in pi_ohern on r917u13n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_239(8162044) in pi_ohern on r917u13n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_240(8162045) in pi_ohern on r917u13n03 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_241(8162046) in pi_ohern on r917u13n03 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_242(8162047) in pi_ohern on r917u13n03 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_243(8162048) in pi_ohern on r917u13n04 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_244(8162049) in pi_ohern on r917u13n04 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_245(8162050) in pi_ohern on r917u29n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_246(8162051) in pi_ohern on r917u29n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_247(8162052) in pi_ohern on r917u29n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_248(8162053) in pi_ohern on r917u29n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_249(8162054) in pi_ohern on r917u29n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_250(8162055) in pi_ohern on r917u29n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_251(8162056) in pi_ohern on r917u29n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_252(8162057) in pi_ohern on r917u29n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_253(8162058) in pi_ohern on r917u29n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_254(8162059) in pi_ohern on r917u29n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_255(8162060) in pi_ohern on r917u29n02 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_256(8162061) in pi_ohern on r917u29n04 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_257(8162062) in pi_ohern on r917u29n04 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_258(8162063) in pi_ohern on r917u29n04 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_259(8162064) in pi_ohern on r815u03n11 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_260(8162065) in pi_ohern on r815u03n12 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_261(8162066) in pi_ohern on r815u03n12 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_262(8162067) in pi_ohern on r815u03n12 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_263(8162068) in pi_ohern on r815u03n12 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_264(8162069) in pi_ohern on r815u03n12 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_265(8162070) in pi_ohern on r911u32n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_266(8162071) in pi_ohern on r911u32n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_267(8162072) in pi_ohern on r807u22n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_268(8162073) in pi_ohern on r807u22n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_269(8162074) in pi_ohern on r807u22n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_270(8162075) in pi_ohern on r807u22n01 Nov 05 00:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_271(8162076) in pi_ohern on r807u22n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_272(8162077) in pi_ohern on r807u22n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_273(8162078) in pi_ohern on r807u24n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_274(8162079) in pi_ohern on r807u24n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_275(8162080) in pi_ohern on r807u24n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_276(8162081) in pi_ohern on r807u24n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_277(8162082) in pi_ohern on r903u12n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_278(8162083) in pi_ohern on r903u24n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_279(8162084) in pi_ohern on r903u24n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_280(8162085) in pi_ohern on r903u24n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_281(8162086) in pi_ohern on r903u24n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_282(8162087) in pi_ohern on r913u32n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_283(8162088) in pi_ohern on r913u32n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_284(8162089) in pi_ohern on r913u35n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_285(8162090) in pi_ohern on r913u35n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_286(8162091) in pi_ohern on r913u35n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_287(8162092) in pi_ohern on r913u35n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_288(8162093) in pi_ohern on r913u35n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_289(8162094) in pi_ohern on r913u35n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_290(8162095) in pi_ohern on r913u35n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_291(8162096) in pi_ohern on r913u35n01 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159923 in day on r815u23n03 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159924 in day on r816u15n04 Nov 05 00:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159925 in day on r816u15n06 Nov 05 00:27:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_47(8162097) in day on r815u03n07 Nov 05 00:27:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3628(8162098) in gpu on r808u17n01 Nov 05 00:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_299(8160370) WEXITSTATUS 0 Nov 05 00:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_299(8160370) done Nov 05 00:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_439(8160757) WEXITSTATUS 0 Nov 05 00:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_439(8160757) done Nov 05 00:27:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 00:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_416(8160734) WEXITSTATUS 0 Nov 05 00:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_416(8160734) done Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5665(8162099) in scavenge on r802u06n04 Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5666(8162100) in scavenge on r802u23n02 Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5667(8162101) in scavenge on r816u35n10 Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5668(8162102) in scavenge on r913u09n03 Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5669(8162103) in scavenge on r816u15n04 Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_278(8160349) WEXITSTATUS 0 Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_278(8160349) done Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_200(8160148) WEXITSTATUS 0 Nov 05 00:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_200(8160148) done Nov 05 00:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_289(8160360) WEXITSTATUS 0 Nov 05 00:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_289(8160360) done Nov 05 00:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_272(8160220) WEXITSTATUS 0 Nov 05 00:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_272(8160220) done Nov 05 00:27:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_422(8160740) WEXITSTATUS 0 Nov 05 00:27:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_422(8160740) done Nov 05 00:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_428(8160746) WEXITSTATUS 0 Nov 05 00:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_428(8160746) done Nov 05 00:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5063(8158368) WEXITSTATUS 1 Nov 05 00:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5063(8158368) done Nov 05 00:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_484(8160924) WEXITSTATUS 0 Nov 05 00:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_484(8160924) done Nov 05 00:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_334(8160405) WEXITSTATUS 0 Nov 05 00:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_334(8160405) done Nov 05 00:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5129(8158469) WEXITSTATUS 1 Nov 05 00:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5129(8158469) done Nov 05 00:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_332(8160403) WEXITSTATUS 0 Nov 05 00:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_332(8160403) done Nov 05 00:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5104(8158437) WEXITSTATUS 1 Nov 05 00:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5104(8158437) done Nov 05 00:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_423(8160741) WEXITSTATUS 0 Nov 05 00:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_423(8160741) done Nov 05 00:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_195(8160143) WEXITSTATUS 0 Nov 05 00:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_195(8160143) done Nov 05 00:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_125(8159953) WEXITSTATUS 0 Nov 05 00:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_125(8159953) done Nov 05 00:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_126(8159954) WEXITSTATUS 0 Nov 05 00:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_126(8159954) done Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_382(8160584) WEXITSTATUS 0 Nov 05 00:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_382(8160584) done Nov 05 00:28:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_199(8160147) WEXITSTATUS 0 Nov 05 00:28:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_199(8160147) done Nov 05 00:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_314(8160385) WEXITSTATUS 0 Nov 05 00:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_314(8160385) done Nov 05 00:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_438(8160756) WEXITSTATUS 0 Nov 05 00:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_438(8160756) done Nov 05 00:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_326(8160397) WEXITSTATUS 0 Nov 05 00:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_326(8160397) done Nov 05 00:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_185(8160013) WEXITSTATUS 0 Nov 05 00:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_185(8160013) done Nov 05 00:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_336(8160407) WEXITSTATUS 0 Nov 05 00:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_336(8160407) done Nov 05 00:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_425(8160743) WEXITSTATUS 0 Nov 05 00:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_425(8160743) done Nov 05 00:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_419(8160737) WEXITSTATUS 0 Nov 05 00:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_419(8160737) done Nov 05 00:28:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1494(8140413) WEXITSTATUS 0 Nov 05 00:28:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1494(8140413) done Nov 05 00:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_356(8160558) WEXITSTATUS 0 Nov 05 00:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_356(8160558) done Nov 05 00:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_220(8160168) WEXITSTATUS 0 Nov 05 00:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_220(8160168) done Nov 05 00:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5517(8160696) WEXITSTATUS 1 Nov 05 00:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5517(8160696) done Nov 05 00:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_293(8160364) WEXITSTATUS 0 Nov 05 00:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_293(8160364) done Nov 05 00:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_347(8160549) WEXITSTATUS 0 Nov 05 00:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_347(8160549) done Nov 05 00:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_298(8160369) WEXITSTATUS 0 Nov 05 00:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_298(8160369) done Nov 05 00:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_215(8160163) WEXITSTATUS 0 Nov 05 00:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_215(8160163) done Nov 05 00:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_322(8160393) WEXITSTATUS 0 Nov 05 00:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_322(8160393) done Nov 05 00:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_239(8160187) WEXITSTATUS 0 Nov 05 00:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_239(8160187) done Nov 05 00:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_216(8160164) WEXITSTATUS 0 Nov 05 00:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_216(8160164) done Nov 05 00:28:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_297(8160368) WEXITSTATUS 0 Nov 05 00:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_297(8160368) done Nov 05 00:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_90(8159858) WEXITSTATUS 0 Nov 05 00:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_90(8159858) done Nov 05 00:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_430(8160748) WEXITSTATUS 0 Nov 05 00:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_430(8160748) done Nov 05 00:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_197(8160145) WEXITSTATUS 0 Nov 05 00:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_197(8160145) done Nov 05 00:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_488(8160928) WEXITSTATUS 0 Nov 05 00:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_488(8160928) done Nov 05 00:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_225(8160173) WEXITSTATUS 0 Nov 05 00:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_225(8160173) done Nov 05 00:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5440(8159655) WEXITSTATUS 1 Nov 05 00:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5440(8159655) done Nov 05 00:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_431(8160749) WEXITSTATUS 0 Nov 05 00:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_431(8160749) done Nov 05 00:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_534(8161091) WEXITSTATUS 0 Nov 05 00:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_534(8161091) done Nov 05 00:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157523 WEXITSTATUS 0 Nov 05 00:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157523 done Nov 05 00:29:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162104 InitPrio=13461 usec=4513 Nov 05 00:29:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_654(8161703) WEXITSTATUS 0 Nov 05 00:29:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_654(8161703) done Nov 05 00:29:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_38(8155893) Nov 05 00:29:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_39(8155894) Nov 05 00:29:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157520 WEXITSTATUS 0 Nov 05 00:29:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157520 done Nov 05 00:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_327(8160398) WEXITSTATUS 0 Nov 05 00:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_327(8160398) done Nov 05 00:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_335(8160406) WEXITSTATUS 0 Nov 05 00:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_335(8160406) done Nov 05 00:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5220(8158607) WEXITSTATUS 1 Nov 05 00:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5220(8158607) done Nov 05 00:29:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_94(8159862) WEXITSTATUS 0 Nov 05 00:29:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_94(8159862) done Nov 05 00:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4807(8157753) WEXITSTATUS 1 Nov 05 00:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4807(8157753) done Nov 05 00:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_295(8160366) WEXITSTATUS 0 Nov 05 00:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_295(8160366) done Nov 05 00:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_342(8160544) WEXITSTATUS 0 Nov 05 00:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_342(8160544) done Nov 05 00:29:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162105 InitPrio=53722 usec=6125 Nov 05 00:29:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162105 NodeList=r801u06n01 #CPUs=1 Partition=education Nov 05 00:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_296(8160367) WEXITSTATUS 0 Nov 05 00:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_296(8160367) done Nov 05 00:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_238(8160186) WEXITSTATUS 0 Nov 05 00:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_238(8160186) done Nov 05 00:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133216 WEXITSTATUS 0 Nov 05 00:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133216 done Nov 05 00:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_426(8160744) WEXITSTATUS 0 Nov 05 00:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_426(8160744) done Nov 05 00:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_237(8160185) WEXITSTATUS 0 Nov 05 00:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_237(8160185) done Nov 05 00:29:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_2(8155909) Nov 05 00:29:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_226(8160174) WEXITSTATUS 0 Nov 05 00:29:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_226(8160174) done Nov 05 00:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_222(8160170) WEXITSTATUS 0 Nov 05 00:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_222(8160170) done Nov 05 00:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_525(8161082) WEXITSTATUS 0 Nov 05 00:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_525(8161082) done Nov 05 00:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2192(8151984) WEXITSTATUS 0 Nov 05 00:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2192(8151984) done Nov 05 00:30:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_217(8160165) WEXITSTATUS 0 Nov 05 00:30:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_217(8160165) done Nov 05 00:30:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5649(8161942) WEXITSTATUS 1 Nov 05 00:30:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5649(8161942) done Nov 05 00:30:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157528 WEXITSTATUS 0 Nov 05 00:30:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157528 done Nov 05 00:30:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5112(8158445) WEXITSTATUS 1 Nov 05 00:30:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5112(8158445) done Nov 05 00:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_487(8160927) WEXITSTATUS 0 Nov 05 00:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_487(8160927) done Nov 05 00:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_362(8160564) WEXITSTATUS 0 Nov 05 00:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_362(8160564) done Nov 05 00:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_371(8160573) WEXITSTATUS 0 Nov 05 00:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_371(8160573) done Nov 05 00:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_353(8160555) WEXITSTATUS 0 Nov 05 00:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_353(8160555) done Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:30:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162104 in pi_balou on r802u30n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_292(8162106) in pi_ohern on r801u32n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_293(8162107) in pi_ohern on r801u32n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_294(8162108) in pi_ohern on r801u32n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_295(8162109) in pi_ohern on r801u32n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_296(8162110) in pi_ohern on r801u32n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_297(8162111) in pi_ohern on r917u13n01 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_298(8162112) in pi_ohern on r917u13n01 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_299(8162113) in pi_ohern on r917u13n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_300(8162114) in pi_ohern on r917u13n03 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_301(8162115) in pi_ohern on r917u13n03 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_302(8162116) in pi_ohern on r917u13n03 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_303(8162117) in pi_ohern on r917u13n03 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_304(8162118) in pi_ohern on r917u13n03 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_305(8162119) in pi_ohern on r917u13n04 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_306(8162120) in pi_ohern on r917u13n04 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_307(8162121) in pi_ohern on r917u29n01 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_308(8162122) in pi_ohern on r917u29n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_309(8162123) in pi_ohern on r917u29n02 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_310(8162124) in pi_ohern on r917u29n03 Nov 05 00:30:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_311(8162125) in pi_ohern on r917u29n03 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_312(8162126) in pi_ohern on r917u29n03 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_313(8162127) in pi_ohern on r917u29n03 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_314(8162128) in pi_ohern on r917u29n04 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_315(8162129) in pi_ohern on r917u29n04 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_316(8162130) in pi_ohern on r917u29n04 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_317(8162131) in pi_ohern on r917u29n04 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_318(8162132) in pi_ohern on r917u29n04 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_319(8162133) in pi_ohern on r917u29n04 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154598 WEXITSTATUS 0 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154598 done Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_294(8160365) WEXITSTATUS 0 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_294(8160365) done Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_320(8162134) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_321(8162135) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_322(8162136) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_323(8162137) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_324(8162138) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_325(8162139) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_326(8162140) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_327(8162141) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_328(8162142) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_329(8162143) in pi_ohern on r815u03n11 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_330(8162144) in pi_ohern on r815u03n12 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_331(8162145) in pi_ohern on r815u03n12 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_332(8162146) in pi_ohern on r815u03n12 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_333(8162147) in pi_ohern on r815u03n12 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_334(8162148) in pi_ohern on r815u03n12 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_335(8162149) in pi_ohern on r903u24n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_336(8162150) in pi_ohern on r807u22n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_337(8162151) in pi_ohern on r807u22n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_338(8162152) in pi_ohern on r807u22n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_339(8162153) in pi_ohern on r807u22n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_340(8162154) in pi_ohern on r807u24n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_341(8162155) in pi_ohern on r807u24n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_342(8162156) in pi_ohern on r807u24n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_343(8162157) in pi_ohern on r807u24n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_344(8162158) in pi_ohern on r807u24n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_345(8162159) in pi_ohern on r913u32n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_346(8162160) in pi_ohern on r913u35n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=43550951 in day on r807u03n04 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r904u07n03 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r904u07n03 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159919 in day on r909u07n01 Nov 05 00:30:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159920 in day on r914u01n02 Nov 05 00:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_48(8162161) in day on r816u03n01 Nov 05 00:30:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_427(8160745) WEXITSTATUS 0 Nov 05 00:30:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_427(8160745) done Nov 05 00:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1100(7572066) WEXITSTATUS 0 Nov 05 00:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1100(7572066) done Nov 05 00:30:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_486(8160926) WEXITSTATUS 0 Nov 05 00:30:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_486(8160926) done Nov 05 00:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_436(8160754) WEXITSTATUS 0 Nov 05 00:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_436(8160754) done Nov 05 00:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_276(8160224) WEXITSTATUS 0 Nov 05 00:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_276(8160224) done Nov 05 00:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_301(8160372) WEXITSTATUS 0 Nov 05 00:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_301(8160372) done Nov 05 00:30:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5670(8162162) in scavenge on r801u04n03 Nov 05 00:30:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5671(8162163) in scavenge on r802u04n04 Nov 05 00:30:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5672(8162164) in scavenge on r816u35n12 Nov 05 00:30:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5673(8162165) in scavenge on r907u24n01 Nov 05 00:30:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5674(8162166) in scavenge on r909u07n01 Nov 05 00:30:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5675(8162167) in scavenge on r909u07n01 Nov 05 00:30:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5676(8162168) in scavenge on r913u13n01 Nov 05 00:30:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5677(8162169) in scavenge on r816u29n03 Nov 05 00:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157527 WEXITSTATUS 0 Nov 05 00:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157527 done Nov 05 00:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_420(8160738) WEXITSTATUS 0 Nov 05 00:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_420(8160738) done Nov 05 00:30:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_417(8160735) WEXITSTATUS 0 Nov 05 00:30:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_417(8160735) done Nov 05 00:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_300(8160371) WEXITSTATUS 0 Nov 05 00:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_300(8160371) done Nov 05 00:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_361(8160563) WEXITSTATUS 0 Nov 05 00:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_361(8160563) done Nov 05 00:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 00:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 00:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_433(8160751) WEXITSTATUS 0 Nov 05 00:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_433(8160751) done Nov 05 00:30:39 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 00:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_283(8160354) WEXITSTATUS 0 Nov 05 00:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_283(8160354) done Nov 05 00:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_386(8160588) WEXITSTATUS 0 Nov 05 00:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_386(8160588) done Nov 05 00:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_223(8160171) WEXITSTATUS 0 Nov 05 00:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_223(8160171) done Nov 05 00:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_227(8160175) WEXITSTATUS 0 Nov 05 00:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_227(8160175) done Nov 05 00:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_348(8160550) WEXITSTATUS 0 Nov 05 00:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_348(8160550) done Nov 05 00:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_221(8160169) WEXITSTATUS 0 Nov 05 00:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_221(8160169) done Nov 05 00:30:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_385(8160587) WEXITSTATUS 0 Nov 05 00:30:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_385(8160587) done Nov 05 00:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=43550951 WEXITSTATUS 0 Nov 05 00:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=43550951 done Nov 05 00:30:59 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=43550951 Nov 05 00:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_284(8160355) WEXITSTATUS 0 Nov 05 00:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_284(8160355) done Nov 05 00:31:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_418(8160736) WEXITSTATUS 0 Nov 05 00:31:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_418(8160736) done Nov 05 00:31:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_390(8160592) WEXITSTATUS 0 Nov 05 00:31:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_390(8160592) done Nov 05 00:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1433(8135966) WEXITSTATUS 0 Nov 05 00:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1433(8135966) done Nov 05 00:31:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_281(8160352) WEXITSTATUS 0 Nov 05 00:31:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_281(8160352) done Nov 05 00:31:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_435(8160753) WEXITSTATUS 0 Nov 05 00:31:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_435(8160753) done Nov 05 00:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_388(8160590) WEXITSTATUS 0 Nov 05 00:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_388(8160590) done Nov 05 00:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_531(8161088) WEXITSTATUS 0 Nov 05 00:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_531(8161088) done Nov 05 00:31:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_541(8161098) WEXITSTATUS 0 Nov 05 00:31:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_541(8161098) done Nov 05 00:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_214(8160162) WEXITSTATUS 0 Nov 05 00:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_214(8160162) done Nov 05 00:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_345(8160547) WEXITSTATUS 0 Nov 05 00:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_345(8160547) done Nov 05 00:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_485(8160925) WEXITSTATUS 0 Nov 05 00:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_485(8160925) done Nov 05 00:31:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2375(8152419) WEXITSTATUS 0 Nov 05 00:31:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2375(8152419) done Nov 05 00:31:39 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:31:40 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_196(8160144) WEXITSTATUS 0 Nov 05 00:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_196(8160144) done Nov 05 00:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_282(8160353) WEXITSTATUS 0 Nov 05 00:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_282(8160353) done Nov 05 00:31:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_359(8160561) WEXITSTATUS 0 Nov 05 00:31:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_359(8160561) done Nov 05 00:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_399(8160717) WEXITSTATUS 0 Nov 05 00:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_399(8160717) done Nov 05 00:31:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157525 WEXITSTATUS 0 Nov 05 00:31:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157525 done Nov 05 00:31:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4758(8157219) WEXITSTATUS 1 Nov 05 00:31:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4758(8157219) done Nov 05 00:31:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_286(8160357) WEXITSTATUS 0 Nov 05 00:31:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_286(8160357) done Nov 05 00:31:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_415(8160733) WEXITSTATUS 0 Nov 05 00:31:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_415(8160733) done Nov 05 00:31:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_316(8160387) WEXITSTATUS 0 Nov 05 00:31:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_316(8160387) done Nov 05 00:32:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_280(8160351) WEXITSTATUS 0 Nov 05 00:32:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_280(8160351) done Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162170 InitPrio=32151 usec=2024 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162171 InitPrio=32151 usec=1409 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162172 InitPrio=32151 usec=2279 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162173 InitPrio=32151 usec=2581 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162174 InitPrio=32151 usec=1625 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162175 InitPrio=32151 usec=2186 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162176 InitPrio=32151 usec=1972 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162177 InitPrio=32151 usec=1464 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162178 InitPrio=32151 usec=1358 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162179 InitPrio=32151 usec=1142 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162180 InitPrio=32151 usec=1010 Nov 05 00:32:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162181 InitPrio=32151 usec=1630 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162182 InitPrio=32151 usec=1176 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162183 InitPrio=32151 usec=2032 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162184 InitPrio=32151 usec=1439 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162185 InitPrio=32151 usec=1100 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162186 InitPrio=32151 usec=1177 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162187 InitPrio=32151 usec=1055 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162188 InitPrio=32151 usec=1126 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162189 InitPrio=32151 usec=1040 Nov 05 00:32:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162190 InitPrio=32151 usec=1104 Nov 05 00:32:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_343(8160545) WEXITSTATUS 0 Nov 05 00:32:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_343(8160545) done Nov 05 00:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_539(8161096) WEXITSTATUS 0 Nov 05 00:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_539(8161096) done Nov 05 00:32:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_424(8160742) WEXITSTATUS 0 Nov 05 00:32:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_424(8160742) done Nov 05 00:32:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131858 WEXITSTATUS 0 Nov 05 00:32:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131858 done Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_421(8160739) WEXITSTATUS 0 Nov 05 00:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_421(8160739) done Nov 05 00:32:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2042(8125820) Nov 05 00:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_351(8160553) WEXITSTATUS 0 Nov 05 00:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_351(8160553) done Nov 05 00:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5084(8158408) WEXITSTATUS 1 Nov 05 00:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5084(8158408) done Nov 05 00:32:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162170 NodeList=r815u23n07 #CPUs=12 Partition=scavenge Nov 05 00:32:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162171 NodeList=r816u03n12 #CPUs=12 Partition=scavenge Nov 05 00:32:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162172 NodeList=r816u03n12 #CPUs=12 Partition=scavenge Nov 05 00:32:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162173 NodeList=r816u29n11 #CPUs=12 Partition=scavenge Nov 05 00:32:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162174 NodeList=r816u35n11 #CPUs=12 Partition=scavenge Nov 05 00:32:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162175 NodeList=r816u35n11 #CPUs=12 Partition=scavenge Nov 05 00:32:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162176 NodeList=r816u35n11 #CPUs=12 Partition=scavenge Nov 05 00:32:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162191 InitPrio=21238 usec=6055 Nov 05 00:32:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_357(8160559) WEXITSTATUS 0 Nov 05 00:32:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_357(8160559) done Nov 05 00:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5519(8160698) WEXITSTATUS 1 Nov 05 00:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5519(8160698) done Nov 05 00:32:43 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:32:44 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:32:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 00:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2195(8151987) WEXITSTATUS 0 Nov 05 00:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2195(8151987) done Nov 05 00:32:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_529(8161086) WEXITSTATUS 0 Nov 05 00:32:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_529(8161086) done Nov 05 00:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3601(8159758) WEXITSTATUS 0 Nov 05 00:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3601(8159758) done Nov 05 00:32:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5250(8158674) WEXITSTATUS 1 Nov 05 00:32:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5250(8158674) done Nov 05 00:32:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_414(8160732) WEXITSTATUS 0 Nov 05 00:32:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_414(8160732) done Nov 05 00:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_367(8160569) WEXITSTATUS 0 Nov 05 00:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_367(8160569) done Nov 05 00:32:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5283(8158731) WEXITSTATUS 1 Nov 05 00:32:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5283(8158731) done Nov 05 00:32:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_340(8160542) WEXITSTATUS 0 Nov 05 00:32:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_340(8160542) done Nov 05 00:32:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8125577 uid 21108 Nov 05 00:33:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5667(8162101) WEXITSTATUS 1 Nov 05 00:33:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5667(8162101) done Nov 05 00:33:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_349(8160551) WEXITSTATUS 0 Nov 05 00:33:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_349(8160551) done Nov 05 00:33:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3628(8162098) WEXITSTATUS 0 Nov 05 00:33:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3628(8162098) done Nov 05 00:33:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5674(8162166) WEXITSTATUS 1 Nov 05 00:33:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5674(8162166) done Nov 05 00:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4609(8156188) WEXITSTATUS 1 Nov 05 00:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4609(8156188) done Nov 05 00:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_38(8159806) WEXITSTATUS 0 Nov 05 00:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_38(8159806) done Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_347(8162192) in pi_ohern on r801u32n02 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_348(8162193) in pi_ohern on r917u13n01 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_349(8162194) in pi_ohern on r917u13n01 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_350(8162195) in pi_ohern on r917u13n01 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_351(8162196) in pi_ohern on r917u13n01 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_352(8162197) in pi_ohern on r917u13n01 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_353(8162198) in pi_ohern on r917u13n01 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_354(8162199) in pi_ohern on r917u13n02 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_355(8162200) in pi_ohern on r917u13n02 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_356(8162201) in pi_ohern on r917u13n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_357(8162202) in pi_ohern on r917u13n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_358(8162203) in pi_ohern on r917u13n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_359(8162204) in pi_ohern on r917u13n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_360(8162205) in pi_ohern on r917u13n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_361(8162206) in pi_ohern on r917u13n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_362(8162207) in pi_ohern on r917u13n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_363(8162208) in pi_ohern on r917u13n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_364(8162209) in pi_ohern on r917u13n04 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_365(8162210) in pi_ohern on r917u29n02 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_366(8162211) in pi_ohern on r917u29n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_367(8162212) in pi_ohern on r917u29n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_368(8162213) in pi_ohern on r917u29n03 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_369(8162214) in pi_ohern on r917u29n04 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_370(8162215) in pi_ohern on r917u29n04 Nov 05 00:33:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_371(8162216) in pi_ohern on r917u29n04 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_372(8162217) in pi_ohern on r917u29n04 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_373(8162218) in pi_ohern on r917u29n04 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_374(8162219) in pi_ohern on r917u29n04 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_375(8162220) in pi_ohern on r917u29n04 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_376(8162221) in pi_ohern on r917u29n04 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_377(8162222) in pi_ohern on r917u29n04 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_378(8162223) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_379(8162224) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_380(8162225) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_381(8162226) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_382(8162227) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_383(8162228) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_384(8162229) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_385(8162230) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_386(8162231) in pi_ohern on r815u03n11 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_387(8162232) in pi_ohern on r815u03n12 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_388(8162233) in pi_ohern on r815u03n12 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_389(8162234) in pi_ohern on r815u03n12 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_390(8162235) in pi_ohern on r815u03n12 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_391(8162236) in pi_ohern on r815u03n12 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_392(8162237) in pi_ohern on r815u03n12 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_393(8162238) in pi_ohern on r815u03n12 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_394(8162239) in pi_ohern on r815u03n12 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_395(8162240) in pi_ohern on r807u22n01 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_396(8162241) in pi_ohern on r807u22n01 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_397(8162242) in pi_ohern on r807u22n01 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_398(8162243) in pi_ohern on r807u22n01 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_399(8162244) in pi_ohern on r903u24n01 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_400(8162245) in pi_ohern on r913u35n01 Nov 05 00:33:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159928 in day on r816u03n03 Nov 05 00:33:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162191 in day on r909u29n03,r918u09n04 Nov 05 00:33:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8158658_49(8158658) in day on r815u09n05 Nov 05 00:33:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3629(8162246) in gpu on r808u11n01 Nov 05 00:33:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3630(8162247) in gpu on r808u17n01 Nov 05 00:33:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162248 InitPrio=23989 usec=3130 Nov 05 00:33:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133386 in gpu on r4513u32n01 Nov 05 00:33:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n03 Nov 05 00:33:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162177 in scavenge on r816u23n10 Nov 05 00:33:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162178 in scavenge on r816u35n10 Nov 05 00:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_437(8160755) WEXITSTATUS 0 Nov 05 00:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_437(8160755) done Nov 05 00:33:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_40(8155914) Nov 05 00:33:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158084 Nov 05 00:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_285(8160356) WEXITSTATUS 0 Nov 05 00:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_285(8160356) done Nov 05 00:33:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5678(8162249) in scavenge on r801u06n01 Nov 05 00:33:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5679(8162250) in scavenge on r802u04n03 Nov 05 00:33:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5680(8162251) in scavenge on r915u35n02 Nov 05 00:33:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5681(8162252) in scavenge on r917u10n04 Nov 05 00:33:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5682(8162253) in scavenge on r808u11n01 Nov 05 00:33:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5683(8162254) in scavenge on r909u07n01 Nov 05 00:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_393(8160711) WEXITSTATUS 0 Nov 05 00:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_393(8160711) done Nov 05 00:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_291(8160362) WEXITSTATUS 0 Nov 05 00:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_291(8160362) done Nov 05 00:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157524 WEXITSTATUS 0 Nov 05 00:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157524 done Nov 05 00:33:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162248 NodeList=r904u01n02 #CPUs=16 Partition=pi_cowles Nov 05 00:33:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162179 NodeList=r816u29n06 #CPUs=12 Partition=scavenge Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1877(8162255) in scavenge on r917u10n04 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1878(8162256) in scavenge on r917u10n04 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:33:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_352(8160554) WEXITSTATUS 0 Nov 05 00:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_352(8160554) done Nov 05 00:33:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4963(8158213) WEXITSTATUS 1 Nov 05 00:33:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4963(8158213) done Nov 05 00:33:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5512(8160515) WEXITSTATUS 1 Nov 05 00:33:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5512(8160515) done Nov 05 00:33:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_369(8160571) WEXITSTATUS 0 Nov 05 00:33:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_369(8160571) done Nov 05 00:33:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_536(8161093) WEXITSTATUS 0 Nov 05 00:33:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_536(8161093) done Nov 05 00:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_524(8161081) WEXITSTATUS 0 Nov 05 00:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_524(8161081) done Nov 05 00:33:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5555(8161045) WEXITSTATUS 1 Nov 05 00:33:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5555(8161045) done Nov 05 00:33:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_344(8160546) WEXITSTATUS 0 Nov 05 00:33:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_344(8160546) done Nov 05 00:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_389(8160591) WEXITSTATUS 0 Nov 05 00:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_389(8160591) done Nov 05 00:33:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_288(8160359) WEXITSTATUS 0 Nov 05 00:33:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_288(8160359) done Nov 05 00:33:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162191 WEXITSTATUS 1 Nov 05 00:33:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162191 done Nov 05 00:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5035(8158331) WEXITSTATUS 1 Nov 05 00:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5035(8158331) done Nov 05 00:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5651(8161944) WEXITSTATUS 1 Nov 05 00:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5651(8161944) done Nov 05 00:33:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_3(8155934) Nov 05 00:33:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_4(8155935) Nov 05 00:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_338(8160540) WEXITSTATUS 0 Nov 05 00:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_338(8160540) done Nov 05 00:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_364(8160566) WEXITSTATUS 0 Nov 05 00:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_364(8160566) done Nov 05 00:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_532(8161089) WEXITSTATUS 0 Nov 05 00:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_532(8161089) done Nov 05 00:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_527(8161084) WEXITSTATUS 0 Nov 05 00:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_527(8161084) done Nov 05 00:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1342(8135480) WEXITSTATUS 0 Nov 05 00:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1342(8135480) done Nov 05 00:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_544(8161101) WEXITSTATUS 0 Nov 05 00:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_544(8161101) done Nov 05 00:34:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_523(8161080) WEXITSTATUS 0 Nov 05 00:34:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_523(8161080) done Nov 05 00:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157526 WEXITSTATUS 0 Nov 05 00:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157526 done Nov 05 00:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_530(8161087) WEXITSTATUS 0 Nov 05 00:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_530(8161087) done Nov 05 00:34:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_412(8160730) WEXITSTATUS 0 Nov 05 00:34:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_412(8160730) done Nov 05 00:34:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_358(8160560) WEXITSTATUS 0 Nov 05 00:34:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_358(8160560) done Nov 05 00:34:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_339(8160541) WEXITSTATUS 0 Nov 05 00:34:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_339(8160541) done Nov 05 00:34:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5683(8162254) WEXITSTATUS 1 Nov 05 00:34:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5683(8162254) done Nov 05 00:34:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_409(8160727) WEXITSTATUS 0 Nov 05 00:34:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_409(8160727) done Nov 05 00:34:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:34:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161727 WEXITSTATUS 0 Nov 05 00:34:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161727 done Nov 05 00:34:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162180 NodeList=r816u23n12 #CPUs=12 Partition=scavenge Nov 05 00:34:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_365(8160567) WEXITSTATUS 0 Nov 05 00:34:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_365(8160567) done Nov 05 00:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_402(8160720) WEXITSTATUS 0 Nov 05 00:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_402(8160720) done Nov 05 00:34:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_292(8160363) WEXITSTATUS 0 Nov 05 00:34:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_292(8160363) done Nov 05 00:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_450(8160890) WEXITSTATUS 0 Nov 05 00:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_450(8160890) done Nov 05 00:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_279(8160350) WEXITSTATUS 0 Nov 05 00:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_279(8160350) done Nov 05 00:34:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_370(8160572) WEXITSTATUS 0 Nov 05 00:34:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_370(8160572) done Nov 05 00:34:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_411(8160729) WEXITSTATUS 0 Nov 05 00:34:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_411(8160729) done Nov 05 00:34:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5653(8161946) WEXITSTATUS 1 Nov 05 00:34:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5653(8161946) done Nov 05 00:34:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_392(8160710) WEXITSTATUS 0 Nov 05 00:34:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_392(8160710) done Nov 05 00:34:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_524(7572693) Nov 05 00:34:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5364(8159174) WEXITSTATUS 1 Nov 05 00:34:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5364(8159174) done Nov 05 00:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4594(8156115) WEXITSTATUS 1 Nov 05 00:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4594(8156115) done Nov 05 00:34:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_368(8160570) WEXITSTATUS 0 Nov 05 00:34:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_368(8160570) done Nov 05 00:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_538(8161095) WEXITSTATUS 0 Nov 05 00:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_538(8161095) done Nov 05 00:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_540(8161097) WEXITSTATUS 0 Nov 05 00:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_540(8161097) done Nov 05 00:35:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_341(8160543) WEXITSTATUS 0 Nov 05 00:35:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_341(8160543) done Nov 05 00:35:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_366(8160568) WEXITSTATUS 0 Nov 05 00:35:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_366(8160568) done Nov 05 00:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4192(8151647) WEXITSTATUS 1 Nov 05 00:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4192(8151647) done Nov 05 00:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5669(8162103) WEXITSTATUS 1 Nov 05 00:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5669(8162103) done Nov 05 00:35:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_337(8160539) WEXITSTATUS 0 Nov 05 00:35:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_337(8160539) done Nov 05 00:35:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5070(8158382) WEXITSTATUS 1 Nov 05 00:35:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5070(8158382) done Nov 05 00:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5386(8159324) WEXITSTATUS 1 Nov 05 00:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5386(8159324) done Nov 05 00:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2291(8152208) WEXITSTATUS 0 Nov 05 00:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2291(8152208) done Nov 05 00:35:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_406(8160724) WEXITSTATUS 0 Nov 05 00:35:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_406(8160724) done Nov 05 00:35:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_489(8160929) WEXITSTATUS 0 Nov 05 00:35:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_489(8160929) done Nov 05 00:35:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162181 NodeList=r816u23n06 #CPUs=12 Partition=scavenge Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_350(8160552) WEXITSTATUS 0 Nov 05 00:35:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_350(8160552) done Nov 05 00:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_355(8160557) WEXITSTATUS 0 Nov 05 00:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_355(8160557) done Nov 05 00:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_372(8160574) WEXITSTATUS 0 Nov 05 00:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_372(8160574) done Nov 05 00:35:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157522 WEXITSTATUS 0 Nov 05 00:35:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157522 done Nov 05 00:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_533(8161090) WEXITSTATUS 0 Nov 05 00:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_533(8161090) done Nov 05 00:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1387(8135688) WEXITSTATUS 0 Nov 05 00:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1387(8135688) done Nov 05 00:35:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_391(8160709) WEXITSTATUS 0 Nov 05 00:35:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_391(8160709) done Nov 05 00:35:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4932(8158119) WEXITSTATUS 1 Nov 05 00:35:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4932(8158119) done Nov 05 00:35:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_491(8160931) WEXITSTATUS 0 Nov 05 00:35:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_491(8160931) done Nov 05 00:35:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5359(8159169) WEXITSTATUS 1 Nov 05 00:35:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5359(8159169) done Nov 05 00:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_363(8160565) WEXITSTATUS 0 Nov 05 00:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_363(8160565) done Nov 05 00:35:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_440(8160758) WEXITSTATUS 0 Nov 05 00:35:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_440(8160758) done Nov 05 00:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3337(8150056) WEXITSTATUS 0 Nov 05 00:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3337(8150056) done Nov 05 00:35:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_537(8161094) WEXITSTATUS 0 Nov 05 00:35:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_537(8161094) done Nov 05 00:36:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_404(8160722) WEXITSTATUS 0 Nov 05 00:36:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_404(8160722) done Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_401(8162257) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_402(8162258) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_403(8162259) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_404(8162260) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_405(8162261) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_406(8162262) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_407(8162263) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_408(8162264) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_409(8162265) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_410(8162266) in pi_ohern on r917u13n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_411(8162267) in pi_ohern on r917u13n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_412(8162268) in pi_ohern on r917u13n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_413(8162269) in pi_ohern on r917u13n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_414(8162270) in pi_ohern on r917u29n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_415(8162271) in pi_ohern on r917u29n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_416(8162272) in pi_ohern on r917u29n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_417(8162273) in pi_ohern on r917u29n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_418(8162274) in pi_ohern on r917u29n02 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_419(8162275) in pi_ohern on r917u29n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_420(8162276) in pi_ohern on r917u29n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_421(8162277) in pi_ohern on r917u29n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_422(8162278) in pi_ohern on r917u29n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_423(8162279) in pi_ohern on r917u29n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_424(8162280) in pi_ohern on r917u29n03 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_425(8162281) in pi_ohern on r917u29n04 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_426(8162282) in pi_ohern on r917u29n04 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_427(8162283) in pi_ohern on r917u29n04 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_428(8162284) in pi_ohern on r917u29n04 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_429(8162285) in pi_ohern on r917u29n04 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_430(8162286) in pi_ohern on r917u29n04 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_431(8162287) in pi_ohern on r917u29n04 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_432(8162288) in pi_ohern on r917u29n04 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_433(8162289) in pi_ohern on r815u03n11 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_434(8162290) in pi_ohern on r815u03n11 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_435(8162291) in pi_ohern on r815u03n12 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_436(8162292) in pi_ohern on r815u03n12 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_437(8162293) in pi_ohern on r815u03n12 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_438(8162294) in pi_ohern on r815u03n12 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_439(8162295) in pi_ohern on r815u03n12 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_440(8162296) in pi_ohern on r815u03n12 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_441(8162297) in pi_ohern on r815u03n12 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_442(8162298) in pi_ohern on r815u03n12 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_443(8162299) in pi_ohern on r903u12n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_444(8162300) in pi_ohern on r911u32n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_445(8162301) in pi_ohern on r913u32n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_446(8162302) in pi_ohern on r807u22n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_447(8162303) in pi_ohern on r807u22n01 Nov 05 00:36:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_448(8162304) in pi_ohern on r807u24n01 Nov 05 00:36:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159926 in day on r918u01n01 Nov 05 00:36:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3631(8162305) in gpu on r903u40n01 Nov 05 00:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_394(8160712) WEXITSTATUS 0 Nov 05 00:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_394(8160712) done Nov 05 00:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1279(8135248) WEXITSTATUS 0 Nov 05 00:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1279(8135248) done Nov 05 00:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_441(8160881) WEXITSTATUS 0 Nov 05 00:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_441(8160881) done Nov 05 00:36:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162182 in scavenge on r816u29n08 Nov 05 00:36:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5630(8161750) WEXITSTATUS 1 Nov 05 00:36:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5630(8161750) done Nov 05 00:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157529 WEXITSTATUS 0 Nov 05 00:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157529 done Nov 05 00:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_535(8161092) WEXITSTATUS 0 Nov 05 00:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_535(8161092) done Nov 05 00:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_508(8161065) WEXITSTATUS 0 Nov 05 00:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_508(8161065) done Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5684(8162306) in scavenge on r801u06n02 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5685(8162307) in scavenge on r904u01n02 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5686(8162308) in scavenge on r915u32n01 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5687(8162309) in scavenge on r802u30n01 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5688(8162310) in scavenge on r802u30n03 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5689(8162311) in scavenge on r805u25n02 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5690(8162312) in scavenge on r909u07n01 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5691(8162313) in scavenge on r913u24n04 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5692(8162314) in scavenge on r919u36n01 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5693(8162315) in scavenge on r815u09n06 Nov 05 00:36:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5694(8162316) in scavenge on r816u15n04 Nov 05 00:36:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3631(8162305) WEXITSTATUS 0 Nov 05 00:36:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3631(8162305) done Nov 05 00:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_403(8160721) WEXITSTATUS 0 Nov 05 00:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_403(8160721) done Nov 05 00:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_478(8160918) WEXITSTATUS 0 Nov 05 00:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_478(8160918) done Nov 05 00:36:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162183 NodeList=r816u23n07 #CPUs=12 Partition=scavenge Nov 05 00:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5641(8161843) WEXITSTATUS 1 Nov 05 00:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5641(8161843) done Nov 05 00:36:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3629(8162246) WEXITSTATUS 0 Nov 05 00:36:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3629(8162246) done Nov 05 00:36:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3550(8158689) WEXITSTATUS 0 Nov 05 00:36:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3550(8158689) done Nov 05 00:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5243(8158650) WEXITSTATUS 1 Nov 05 00:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5243(8158650) done Nov 05 00:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_606(8161289) WEXITSTATUS 0 Nov 05 00:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_606(8161289) done Nov 05 00:36:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_360(8160562) WEXITSTATUS 0 Nov 05 00:36:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_360(8160562) done Nov 05 00:36:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1341(8135479) WEXITSTATUS 0 Nov 05 00:36:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1341(8135479) done Nov 05 00:36:53 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3834(8144448) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:36:53 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157699 had to be killed Nov 05 00:36:53 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5622(8161723) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:36:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3834(8144448) Nov 05 00:36:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5622(8161723) Nov 05 00:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_449(8160889) WEXITSTATUS 0 Nov 05 00:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_449(8160889) done Nov 05 00:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_451(8160891) WEXITSTATUS 0 Nov 05 00:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_451(8160891) done Nov 05 00:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_490(8160930) WEXITSTATUS 0 Nov 05 00:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_490(8160930) done Nov 05 00:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_455(8160895) WEXITSTATUS 0 Nov 05 00:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_455(8160895) done Nov 05 00:37:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157521 WEXITSTATUS 0 Nov 05 00:37:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157521 done Nov 05 00:37:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4331(8154395) WEXITSTATUS 1 Nov 05 00:37:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4331(8154395) done Nov 05 00:37:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_598(8161281) WEXITSTATUS 0 Nov 05 00:37:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_598(8161281) done Nov 05 00:37:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1283(8135262) WEXITSTATUS 0 Nov 05 00:37:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1283(8135262) done Nov 05 00:37:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162317 InitPrio=32151 usec=1714 Nov 05 00:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5421(8159466) WEXITSTATUS 1 Nov 05 00:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5421(8159466) done Nov 05 00:37:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_401(8160719) WEXITSTATUS 0 Nov 05 00:37:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_401(8160719) done Nov 05 00:37:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_287(8160358) WEXITSTATUS 0 Nov 05 00:37:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_287(8160358) done Nov 05 00:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159114 WEXITSTATUS 0 Nov 05 00:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159114 done Nov 05 00:37:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_41(8155949) Nov 05 00:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5469(8159892) WEXITSTATUS 1 Nov 05 00:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5469(8159892) done Nov 05 00:37:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2365(8152409) WEXITSTATUS 0 Nov 05 00:37:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2365(8152409) done Nov 05 00:37:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_472(8160912) WEXITSTATUS 0 Nov 05 00:37:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_472(8160912) done Nov 05 00:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_346(8160548) WEXITSTATUS 0 Nov 05 00:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_346(8160548) done Nov 05 00:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161697 WEXITSTATUS 0 Nov 05 00:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161697 done Nov 05 00:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5459(8159775) WEXITSTATUS 1 Nov 05 00:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5459(8159775) done Nov 05 00:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162184 NodeList=r816u29n08 #CPUs=12 Partition=scavenge Nov 05 00:37:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_395(8160713) WEXITSTATUS 0 Nov 05 00:37:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_395(8160713) done Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:37:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:37:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159113 WEXITSTATUS 0 Nov 05 00:37:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159113 done Nov 05 00:37:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_492(8160932) WEXITSTATUS 0 Nov 05 00:37:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_492(8160932) done Nov 05 00:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_396(8160714) WEXITSTATUS 0 Nov 05 00:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_396(8160714) done Nov 05 00:37:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_542(8161099) WEXITSTATUS 0 Nov 05 00:37:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_542(8161099) done Nov 05 00:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_493(8160933) WEXITSTATUS 0 Nov 05 00:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_493(8160933) done Nov 05 00:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131856 WEXITSTATUS 0 Nov 05 00:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131856 done Nov 05 00:37:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_467(8160907) WEXITSTATUS 0 Nov 05 00:37:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_467(8160907) done Nov 05 00:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_457(8160897) WEXITSTATUS 0 Nov 05 00:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_457(8160897) done Nov 05 00:37:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4730(8156859) WEXITSTATUS 1 Nov 05 00:37:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4730(8156859) done Nov 05 00:37:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_548(8161105) WEXITSTATUS 0 Nov 05 00:37:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_548(8161105) done Nov 05 00:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159117 WEXITSTATUS 0 Nov 05 00:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159117 done Nov 05 00:37:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_460(8160900) WEXITSTATUS 0 Nov 05 00:37:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_460(8160900) done Nov 05 00:37:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162318 InitPrio=32151 usec=1905 Nov 05 00:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_482(8160922) WEXITSTATUS 0 Nov 05 00:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_482(8160922) done Nov 05 00:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_466(8160906) WEXITSTATUS 0 Nov 05 00:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_466(8160906) done Nov 05 00:38:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_543(8161100) WEXITSTATUS 0 Nov 05 00:38:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_543(8161100) done Nov 05 00:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5547(8161037) WEXITSTATUS 1 Nov 05 00:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5547(8161037) done Nov 05 00:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_509(8161066) WEXITSTATUS 0 Nov 05 00:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_509(8161066) done Nov 05 00:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1252(8135154) WEXITSTATUS 0 Nov 05 00:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1252(8135154) done Nov 05 00:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_408(8160726) WEXITSTATUS 0 Nov 05 00:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_408(8160726) done Nov 05 00:38:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4775(8157457) WEXITSTATUS 1 Nov 05 00:38:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4775(8157457) done Nov 05 00:38:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_410(8160728) WEXITSTATUS 0 Nov 05 00:38:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_410(8160728) done Nov 05 00:38:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5624(8161725) WEXITSTATUS 1 Nov 05 00:38:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5624(8161725) done Nov 05 00:38:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159116 WEXITSTATUS 0 Nov 05 00:38:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159116 done Nov 05 00:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_400(8160718) WEXITSTATUS 0 Nov 05 00:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_400(8160718) done Nov 05 00:38:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_497(8161054) WEXITSTATUS 0 Nov 05 00:38:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_497(8161054) done Nov 05 00:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162104 WEXITSTATUS 0 Nov 05 00:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162104 done Nov 05 00:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5343(8159021) WEXITSTATUS 1 Nov 05 00:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5343(8159021) done Nov 05 00:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8143894_9(8144554) WEXITSTATUS 0 Nov 05 00:38:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8143894_9(8144554) done Nov 05 00:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_518(8161075) WEXITSTATUS 0 Nov 05 00:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_518(8161075) done Nov 05 00:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5204(8158567) WEXITSTATUS 1 Nov 05 00:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5204(8158567) done Nov 05 00:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_526(8161083) WEXITSTATUS 0 Nov 05 00:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_526(8161083) done Nov 05 00:38:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5108(8158441) WEXITSTATUS 1 Nov 05 00:38:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5108(8158441) done Nov 05 00:38:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_443(8160883) WEXITSTATUS 0 Nov 05 00:38:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_443(8160883) done Nov 05 00:38:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_481(8160921) WEXITSTATUS 0 Nov 05 00:38:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_481(8160921) done Nov 05 00:38:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_405(8160723) WEXITSTATUS 0 Nov 05 00:38:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_405(8160723) done Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_449(8162319) in pi_ohern on r917u13n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_450(8162320) in pi_ohern on r917u13n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_451(8162321) in pi_ohern on r917u13n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_452(8162322) in pi_ohern on r917u13n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_453(8162323) in pi_ohern on r917u13n02 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_454(8162324) in pi_ohern on r917u13n02 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_455(8162325) in pi_ohern on r917u13n03 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_456(8162326) in pi_ohern on r917u13n04 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_457(8162327) in pi_ohern on r917u13n04 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_458(8162328) in pi_ohern on r917u13n04 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_459(8162329) in pi_ohern on r917u29n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_460(8162330) in pi_ohern on r917u29n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_461(8162331) in pi_ohern on r917u29n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_462(8162332) in pi_ohern on r917u29n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_463(8162333) in pi_ohern on r917u29n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_464(8162334) in pi_ohern on r917u29n02 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_465(8162335) in pi_ohern on r917u29n02 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_466(8162336) in pi_ohern on r917u29n02 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_467(8162337) in pi_ohern on r917u29n02 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_468(8162338) in pi_ohern on r917u29n02 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_469(8162339) in pi_ohern on r917u29n04 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_470(8162340) in pi_ohern on r917u29n04 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_471(8162341) in pi_ohern on r917u29n04 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_472(8162342) in pi_ohern on r917u29n04 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_473(8162343) in pi_ohern on r815u03n12 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_474(8162344) in pi_ohern on r815u03n12 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_475(8162345) in pi_ohern on r911u32n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_476(8162346) in pi_ohern on r807u22n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_477(8162347) in pi_ohern on r807u24n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_478(8162348) in pi_ohern on r807u24n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_479(8162349) in pi_ohern on r903u12n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_480(8162350) in pi_ohern on r903u26n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_481(8162351) in pi_ohern on r913u32n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_482(8162352) in pi_ohern on r913u35n01 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159930 in day on r815u23n06 Nov 05 00:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161675 in day on r816u15n02 Nov 05 00:38:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3632(8162353) in gpu on r903u34n01 Nov 05 00:38:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3633(8162354) in gpu on r903u40n01 Nov 05 00:38:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3634(8162355) in gpu on r808u08n01 Nov 05 00:38:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154615 WEXITSTATUS 0 Nov 05 00:38:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154615 done Nov 05 00:39:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1879(8162356) in day on r815u09n02 Nov 05 00:39:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133387 in gpu on r808u11n01 Nov 05 00:39:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 00:39:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162185 in scavenge on r815u03n09 Nov 05 00:39:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_480(8160920) WEXITSTATUS 0 Nov 05 00:39:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_480(8160920) done Nov 05 00:39:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5351(8159085) WEXITSTATUS 1 Nov 05 00:39:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5351(8159085) done Nov 05 00:39:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5682(8162253) WEXITSTATUS 1 Nov 05 00:39:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5682(8162253) done Nov 05 00:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_397(8160715) WEXITSTATUS 0 Nov 05 00:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_397(8160715) done Nov 05 00:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_444(8160884) WEXITSTATUS 0 Nov 05 00:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_444(8160884) done Nov 05 00:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_477(8160917) WEXITSTATUS 0 Nov 05 00:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_477(8160917) done Nov 05 00:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_445(8160885) WEXITSTATUS 0 Nov 05 00:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_445(8160885) done Nov 05 00:39:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_608(8161291) WEXITSTATUS 0 Nov 05 00:39:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_608(8161291) done Nov 05 00:39:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5695(8162357) in scavenge on r802u11n03 Nov 05 00:39:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5696(8162358) in scavenge on r903u34n01 Nov 05 00:39:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5697(8162359) in scavenge on r805u06n03 Nov 05 00:39:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5698(8162360) in scavenge on r807u11n04 Nov 05 00:39:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5699(8162361) in scavenge on r918u32n04 Nov 05 00:39:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5700(8162362) in scavenge on r913u09n03 Nov 05 00:39:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5701(8162363) in scavenge on r913u24n04 Nov 05 00:39:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5702(8162364) in scavenge on r816u03n05 Nov 05 00:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_506(8161063) WEXITSTATUS 0 Nov 05 00:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_506(8161063) done Nov 05 00:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1351(8135504) WEXITSTATUS 0 Nov 05 00:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1351(8135504) done Nov 05 00:39:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_185(7572353) in scavenge on r815u23n06 Nov 05 00:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_511(8161068) WEXITSTATUS 0 Nov 05 00:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_511(8161068) done Nov 05 00:39:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5560(8161214) WEXITSTATUS 1 Nov 05 00:39:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5560(8161214) done Nov 05 00:39:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_550(8161107) WEXITSTATUS 0 Nov 05 00:39:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_550(8161107) done Nov 05 00:39:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_448(8160888) WEXITSTATUS 0 Nov 05 00:39:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_448(8160888) done Nov 05 00:39:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159115 WEXITSTATUS 0 Nov 05 00:39:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159115 done Nov 05 00:39:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_603(8161286) WEXITSTATUS 0 Nov 05 00:39:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_603(8161286) done Nov 05 00:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4699(8156471) WEXITSTATUS 1 Nov 05 00:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4699(8156471) done Nov 05 00:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4772(8157454) WEXITSTATUS 1 Nov 05 00:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4772(8157454) done Nov 05 00:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_528(8161085) WEXITSTATUS 0 Nov 05 00:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_528(8161085) done Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4974(8158236) WEXITSTATUS 1 Nov 05 00:39:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4974(8158236) done Nov 05 00:39:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_469(8160909) WEXITSTATUS 0 Nov 05 00:39:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_469(8160909) done Nov 05 00:39:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_453(8160893) WEXITSTATUS 0 Nov 05 00:39:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_453(8160893) done Nov 05 00:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1413(8135946) WEXITSTATUS 0 Nov 05 00:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1413(8135946) done Nov 05 00:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1533(8145780) WEXITSTATUS 0 Nov 05 00:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1533(8145780) done Nov 05 00:39:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_463(8160903) WEXITSTATUS 0 Nov 05 00:39:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_463(8160903) done Nov 05 00:39:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_462(8160902) WEXITSTATUS 0 Nov 05 00:39:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_462(8160902) done Nov 05 00:39:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_569(8161252) WEXITSTATUS 0 Nov 05 00:39:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_569(8161252) done Nov 05 00:39:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162365 InitPrio=59533 usec=4742 Nov 05 00:39:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162366 InitPrio=59533 usec=4698 Nov 05 00:39:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162367 InitPrio=59533 usec=3383 Nov 05 00:39:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162368 InitPrio=59533 usec=4677 Nov 05 00:39:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162365 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:39:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162366 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:39:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162367 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:39:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162368 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_452(8160892) WEXITSTATUS 0 Nov 05 00:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_452(8160892) done Nov 05 00:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_468(8160908) WEXITSTATUS 0 Nov 05 00:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_468(8160908) done Nov 05 00:39:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162366 WEXITSTATUS 0 Nov 05 00:39:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162366 done Nov 05 00:39:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162365 WEXITSTATUS 0 Nov 05 00:39:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162365 done Nov 05 00:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_551(8161108) WEXITSTATUS 0 Nov 05 00:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_551(8161108) done Nov 05 00:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162368 WEXITSTATUS 0 Nov 05 00:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162368 done Nov 05 00:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162367 WEXITSTATUS 0 Nov 05 00:39:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162367 done Nov 05 00:39:54 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4098(8148578) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:39:54 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5558(8161048) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:39:54 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5620(8161721) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:39:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5620(8161721) Nov 05 00:39:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5558(8161048) Nov 05 00:39:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4098(8148578) Nov 05 00:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5635(8161837) WEXITSTATUS 1 Nov 05 00:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5635(8161837) done Nov 05 00:40:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_494(8160934) WEXITSTATUS 0 Nov 05 00:40:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_494(8160934) done Nov 05 00:40:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_501(8161058) WEXITSTATUS 0 Nov 05 00:40:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_501(8161058) done Nov 05 00:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_545(8161102) WEXITSTATUS 0 Nov 05 00:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_545(8161102) done Nov 05 00:40:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157531 WEXITSTATUS 0 Nov 05 00:40:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157531 done Nov 05 00:40:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_505(8161062) WEXITSTATUS 0 Nov 05 00:40:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_505(8161062) done Nov 05 00:40:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5393(8159331) WEXITSTATUS 1 Nov 05 00:40:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5393(8159331) done Nov 05 00:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_546(8161103) WEXITSTATUS 0 Nov 05 00:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_546(8161103) done Nov 05 00:40:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5645(8161938) WEXITSTATUS 1 Nov 05 00:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5645(8161938) done Nov 05 00:40:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8155891 uid 17477 Nov 05 00:40:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162186 NodeList=r815u15n11 #CPUs=12 Partition=scavenge Nov 05 00:40:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_503(8161060) WEXITSTATUS 0 Nov 05 00:40:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_503(8161060) done Nov 05 00:40:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_596(8161279) WEXITSTATUS 0 Nov 05 00:40:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_596(8161279) done Nov 05 00:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_549(8161106) WEXITSTATUS 0 Nov 05 00:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_549(8161106) done Nov 05 00:40:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_475(8160915) WEXITSTATUS 0 Nov 05 00:40:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_475(8160915) done Nov 05 00:40:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_479(8160919) WEXITSTATUS 0 Nov 05 00:40:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_479(8160919) done Nov 05 00:40:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_580(8161263) WEXITSTATUS 0 Nov 05 00:40:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_580(8161263) done Nov 05 00:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133215 WEXITSTATUS 0 Nov 05 00:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133215 done Nov 05 00:40:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_474(8160914) WEXITSTATUS 0 Nov 05 00:40:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_474(8160914) done Nov 05 00:40:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_547(8161104) WEXITSTATUS 0 Nov 05 00:40:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_547(8161104) done Nov 05 00:40:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4692(8156464) WEXITSTATUS 1 Nov 05 00:40:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4692(8156464) done Nov 05 00:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_564(8161247) WEXITSTATUS 0 Nov 05 00:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_564(8161247) done Nov 05 00:41:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_413(8160731) WEXITSTATUS 0 Nov 05 00:41:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_413(8160731) done Nov 05 00:41:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_496(8161053) WEXITSTATUS 0 Nov 05 00:41:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_496(8161053) done Nov 05 00:41:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5532(8160867) WEXITSTATUS 1 Nov 05 00:41:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5532(8160867) done Nov 05 00:41:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_617(8161414) WEXITSTATUS 0 Nov 05 00:41:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_617(8161414) done Nov 05 00:41:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:41:16 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:41:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_354(8160556) WEXITSTATUS 0 Nov 05 00:41:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_354(8160556) done Nov 05 00:41:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_605(8161288) WEXITSTATUS 0 Nov 05 00:41:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_605(8161288) done Nov 05 00:41:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_574(8161257) WEXITSTATUS 0 Nov 05 00:41:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_574(8161257) done Nov 05 00:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_575(8161258) WEXITSTATUS 0 Nov 05 00:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_575(8161258) done Nov 05 00:41:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_446(8160886) WEXITSTATUS 0 Nov 05 00:41:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_446(8160886) done Nov 05 00:41:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1480(8138594) WEXITSTATUS 0 Nov 05 00:41:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1480(8138594) done Nov 05 00:41:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161848 WEXITSTATUS 0 Nov 05 00:41:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161848 done Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:41:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_407(8160725) WEXITSTATUS 0 Nov 05 00:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_407(8160725) done Nov 05 00:41:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1445(8136304) WEXITSTATUS 0 Nov 05 00:41:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1445(8136304) done Nov 05 00:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157532 WEXITSTATUS 0 Nov 05 00:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157532 done Nov 05 00:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_447(8160887) WEXITSTATUS 0 Nov 05 00:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_447(8160887) done Nov 05 00:41:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5379(8159272) WEXITSTATUS 1 Nov 05 00:41:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5379(8159272) done Nov 05 00:41:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5374(8159267) WEXITSTATUS 1 Nov 05 00:41:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5374(8159267) done Nov 05 00:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_645(8161442) WEXITSTATUS 0 Nov 05 00:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_645(8161442) done Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_483(8162369) in pi_ohern on r917u13n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_484(8162370) in pi_ohern on r917u13n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_485(8162371) in pi_ohern on r917u13n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_486(8162372) in pi_ohern on r917u13n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_487(8162373) in pi_ohern on r917u13n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_488(8162374) in pi_ohern on r917u13n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_489(8162375) in pi_ohern on r917u13n02 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_490(8162376) in pi_ohern on r917u13n02 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_491(8162377) in pi_ohern on r917u13n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_492(8162378) in pi_ohern on r917u13n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_493(8162379) in pi_ohern on r917u13n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_494(8162380) in pi_ohern on r917u13n04 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_495(8162381) in pi_ohern on r917u13n04 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_496(8162382) in pi_ohern on r917u13n04 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_497(8162383) in pi_ohern on r917u13n04 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_498(8162384) in pi_ohern on r917u13n04 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_499(8162385) in pi_ohern on r917u29n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_500(8162386) in pi_ohern on r917u29n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_501(8162387) in pi_ohern on r917u29n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_502(8162388) in pi_ohern on r917u29n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_503(8162389) in pi_ohern on r917u29n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_504(8162390) in pi_ohern on r917u29n01 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_505(8162391) in pi_ohern on r917u29n02 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_506(8162392) in pi_ohern on r917u29n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_507(8162393) in pi_ohern on r917u29n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_508(8162394) in pi_ohern on r917u29n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_509(8162395) in pi_ohern on r917u29n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_510(8162396) in pi_ohern on r917u29n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_511(8162397) in pi_ohern on r917u29n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_512(8162398) in pi_ohern on r917u29n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_513(8162399) in pi_ohern on r917u29n03 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_644(8161441) WEXITSTATUS 0 Nov 05 00:41:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_644(8161441) done Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_514(8162400) in pi_ohern on r917u29n03 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_515(8162401) in pi_ohern on r917u29n04 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_516(8162402) in pi_ohern on r917u29n04 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_517(8162403) in pi_ohern on r917u29n04 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_518(8162404) in pi_ohern on r917u29n04 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_519(8162405) in pi_ohern on r917u29n04 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_520(8162406) in pi_ohern on r917u29n04 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_521(8162407) in pi_ohern on r815u03n12 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_522(8162408) in pi_ohern on r807u22n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_523(8162409) in pi_ohern on r903u26n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_524(8162410) in pi_ohern on r903u26n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_525(8162411) in pi_ohern on r903u26n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_526(8162412) in pi_ohern on r911u32n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_527(8162413) in pi_ohern on r913u32n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_528(8162414) in pi_ohern on r913u35n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_529(8162415) in pi_ohern on r913u35n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159931 in day on r815u29n03 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159932 in day on r816u09n02 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159933 in day on r816u15n01 Nov 05 00:41:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161676 in day on r816u15n06 Nov 05 00:41:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_454(8160894) WEXITSTATUS 0 Nov 05 00:41:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_454(8160894) done Nov 05 00:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1452(8136672) WEXITSTATUS 0 Nov 05 00:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1452(8136672) done Nov 05 00:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162187 in scavenge on r816u29n07 Nov 05 00:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_512(8161069) WEXITSTATUS 0 Nov 05 00:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_512(8161069) done Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5703(8162416) in scavenge on r906u01n02 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5704(8162417) in scavenge on r801u27n02 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5705(8162418) in scavenge on r802u11n02 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5706(8162419) in scavenge on r903u04n01 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5707(8162420) in scavenge on r911u05n01 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5708(8162421) in scavenge on r915u35n04 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5709(8162422) in scavenge on r916u04n01 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5710(8162423) in scavenge on r917u10n03 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5711(8162424) in scavenge on r808u11n01 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5712(8162425) in scavenge on r805u02n04 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5713(8162426) in scavenge on r805u23n04 Nov 05 00:41:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5714(8162427) in scavenge on r816u03n04 Nov 05 00:42:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_513(8161070) WEXITSTATUS 0 Nov 05 00:42:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_513(8161070) done Nov 05 00:42:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_385(7572553) in scavenge on r4513u26n01 Nov 05 00:42:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_582(8161265) WEXITSTATUS 0 Nov 05 00:42:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_582(8161265) done Nov 05 00:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_510(8161067) WEXITSTATUS 0 Nov 05 00:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_510(8161067) done Nov 05 00:42:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_552(8161109) WEXITSTATUS 0 Nov 05 00:42:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_552(8161109) done Nov 05 00:42:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1880(8162428) in scavenge on r816u23n04 Nov 05 00:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_507(8161064) WEXITSTATUS 0 Nov 05 00:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_507(8161064) done Nov 05 00:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_465(8160905) WEXITSTATUS 0 Nov 05 00:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_465(8160905) done Nov 05 00:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_398(8160716) WEXITSTATUS 0 Nov 05 00:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_398(8160716) done Nov 05 00:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_476(8160916) WEXITSTATUS 0 Nov 05 00:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_476(8160916) done Nov 05 00:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157530 WEXITSTATUS 0 Nov 05 00:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157530 done Nov 05 00:42:25 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4532(8155933) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:42:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5042(8158338) WEXITSTATUS 1 Nov 05 00:42:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5042(8158338) done Nov 05 00:42:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_498(8161055) WEXITSTATUS 0 Nov 05 00:42:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_498(8161055) done Nov 05 00:42:27 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4532(8155933) Nov 05 00:42:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162188 NodeList=r816u03n10 #CPUs=12 Partition=scavenge Nov 05 00:42:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_568(8161251) WEXITSTATUS 0 Nov 05 00:42:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_568(8161251) done Nov 05 00:42:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162429 InitPrio=21238 usec=6253 Nov 05 00:42:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162430 InitPrio=59533 usec=3710 Nov 05 00:42:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162431 InitPrio=59533 usec=3373 Nov 05 00:42:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162432 InitPrio=59533 usec=2208 Nov 05 00:42:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162433 InitPrio=59533 usec=3943 Nov 05 00:42:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162430 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:42:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162431 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 00:42:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162432 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:42:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162433 NodeList=r816u15n10 #CPUs=1 Partition=pi_anticevic Nov 05 00:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_456(8160896) WEXITSTATUS 0 Nov 05 00:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_456(8160896) done Nov 05 00:42:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157535 WEXITSTATUS 0 Nov 05 00:42:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157535 done Nov 05 00:42:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_611(8161294) WEXITSTATUS 0 Nov 05 00:42:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_611(8161294) done Nov 05 00:43:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_495(8161052) WEXITSTATUS 0 Nov 05 00:43:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_495(8161052) done Nov 05 00:43:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_499(8161056) WEXITSTATUS 0 Nov 05 00:43:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_499(8161056) done Nov 05 00:43:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_458(8160898) WEXITSTATUS 0 Nov 05 00:43:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_458(8160898) done Nov 05 00:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_555(8161238) WEXITSTATUS 0 Nov 05 00:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_555(8161238) done Nov 05 00:43:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_601(8161284) WEXITSTATUS 0 Nov 05 00:43:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_601(8161284) done Nov 05 00:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_514(8161071) WEXITSTATUS 0 Nov 05 00:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_514(8161071) done Nov 05 00:43:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_599(8161282) WEXITSTATUS 0 Nov 05 00:43:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_599(8161282) done Nov 05 00:43:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_557(8161240) WEXITSTATUS 0 Nov 05 00:43:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_557(8161240) done Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_593(8161276) WEXITSTATUS 0 Nov 05 00:43:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_593(8161276) done Nov 05 00:43:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_558(8161241) WEXITSTATUS 0 Nov 05 00:43:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_558(8161241) done Nov 05 00:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_560(8161243) WEXITSTATUS 0 Nov 05 00:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_560(8161243) done Nov 05 00:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3634(8162355) WEXITSTATUS 0 Nov 05 00:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3634(8162355) done Nov 05 00:43:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5287(8158735) WEXITSTATUS 1 Nov 05 00:43:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5287(8158735) done Nov 05 00:43:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_567(8161250) WEXITSTATUS 0 Nov 05 00:43:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_567(8161250) done Nov 05 00:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_613(8161410) WEXITSTATUS 0 Nov 05 00:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_613(8161410) done Nov 05 00:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_570(8161253) WEXITSTATUS 0 Nov 05 00:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_570(8161253) done Nov 05 00:43:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_565(8161248) WEXITSTATUS 0 Nov 05 00:43:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_565(8161248) done Nov 05 00:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2177(8151969) WEXITSTATUS 0 Nov 05 00:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2177(8151969) done Nov 05 00:43:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1400(8135842) WEXITSTATUS 0 Nov 05 00:43:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1400(8135842) done Nov 05 00:43:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_571(8161254) WEXITSTATUS 0 Nov 05 00:43:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_571(8161254) done Nov 05 00:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3630(8162247) WEXITSTATUS 0 Nov 05 00:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3630(8162247) done Nov 05 00:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_519(8161076) WEXITSTATUS 0 Nov 05 00:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_519(8161076) done Nov 05 00:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_500(8161057) WEXITSTATUS 0 Nov 05 00:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_500(8161057) done Nov 05 00:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_459(8160899) WEXITSTATUS 0 Nov 05 00:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_459(8160899) done Nov 05 00:44:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_604(8161287) WEXITSTATUS 0 Nov 05 00:44:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_604(8161287) done Nov 05 00:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_566(8161249) WEXITSTATUS 0 Nov 05 00:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_566(8161249) done Nov 05 00:44:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3633(8162354) WEXITSTATUS 0 Nov 05 00:44:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3633(8162354) done Nov 05 00:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_609(8161292) WEXITSTATUS 0 Nov 05 00:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_609(8161292) done Nov 05 00:44:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8154263 Nov 05 00:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_614(8161411) WEXITSTATUS 0 Nov 05 00:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_614(8161411) done Nov 05 00:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_607(8161290) WEXITSTATUS 0 Nov 05 00:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_607(8161290) done Nov 05 00:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_646(8161443) WEXITSTATUS 0 Nov 05 00:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_646(8161443) done Nov 05 00:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154600 WEXITSTATUS 0 Nov 05 00:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154600 done Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_530(8162434) in pi_ohern on r917u13n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_531(8162435) in pi_ohern on r917u13n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_532(8162436) in pi_ohern on r917u13n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_533(8162437) in pi_ohern on r917u13n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_534(8162438) in pi_ohern on r917u13n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_535(8162439) in pi_ohern on r917u13n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_536(8162440) in pi_ohern on r917u13n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_537(8162441) in pi_ohern on r917u13n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_538(8162442) in pi_ohern on r917u13n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_539(8162443) in pi_ohern on r917u13n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_540(8162444) in pi_ohern on r917u13n03 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_541(8162445) in pi_ohern on r917u13n03 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_542(8162446) in pi_ohern on r917u13n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_543(8162447) in pi_ohern on r917u13n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_544(8162448) in pi_ohern on r917u13n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_545(8162449) in pi_ohern on r917u13n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_546(8162450) in pi_ohern on r917u13n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_547(8162451) in pi_ohern on r917u13n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_548(8162452) in pi_ohern on r917u13n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_549(8162453) in pi_ohern on r917u13n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_550(8162454) in pi_ohern on r917u29n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_551(8162455) in pi_ohern on r917u29n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_552(8162456) in pi_ohern on r917u29n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_553(8162457) in pi_ohern on r917u29n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_554(8162458) in pi_ohern on r917u29n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_555(8162459) in pi_ohern on r917u29n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_556(8162460) in pi_ohern on r917u29n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_557(8162461) in pi_ohern on r917u29n02 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_558(8162462) in pi_ohern on r917u29n03 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_559(8162463) in pi_ohern on r917u29n03 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_560(8162464) in pi_ohern on r917u29n04 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_561(8162465) in pi_ohern on r807u24n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_562(8162466) in pi_ohern on r807u24n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_563(8162467) in pi_ohern on r807u24n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_564(8162468) in pi_ohern on r903u12n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_565(8162469) in pi_ohern on r903u12n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_566(8162470) in pi_ohern on r903u12n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_567(8162471) in pi_ohern on r903u12n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_568(8162472) in pi_ohern on r903u12n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_569(8162473) in pi_ohern on r903u26n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_570(8162474) in pi_ohern on r903u26n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_571(8162475) in pi_ohern on r913u32n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_572(8162476) in pi_ohern on r913u35n01 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5479(8160128) WEXITSTATUS 1 Nov 05 00:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5479(8160128) done Nov 05 00:44:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159934 in day on r815u15n07 Nov 05 00:44:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159935 in day on r816u03n08 Nov 05 00:44:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162429 in day on r909u29n03,r918u09n04 Nov 05 00:44:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3635(8162477) in gpu on r903u40n01 Nov 05 00:44:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3636(8162478) in gpu on r808u08n01 Nov 05 00:44:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3637(8162479) in gpu on r808u17n01 Nov 05 00:44:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n02 Nov 05 00:44:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162189 in scavenge on r815u29n05 Nov 05 00:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_572(8161255) WEXITSTATUS 0 Nov 05 00:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_572(8161255) done Nov 05 00:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_616(8161413) WEXITSTATUS 0 Nov 05 00:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_616(8161413) done Nov 05 00:44:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_461(8160901) WEXITSTATUS 0 Nov 05 00:44:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_461(8160901) done Nov 05 00:44:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_553(8161110) WEXITSTATUS 0 Nov 05 00:44:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_553(8161110) done Nov 05 00:44:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5715(8162480) in scavenge on r805u11n01 Nov 05 00:44:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5716(8162481) in scavenge on r904u04n01 Nov 05 00:44:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5717(8162482) in scavenge on r815u15n07 Nov 05 00:44:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5475(8159898) WEXITSTATUS 1 Nov 05 00:44:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5475(8159898) done Nov 05 00:44:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_504(8161061) WEXITSTATUS 0 Nov 05 00:44:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_504(8161061) done Nov 05 00:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_483(8160923) WEXITSTATUS 0 Nov 05 00:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_483(8160923) done Nov 05 00:44:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_578(8161261) WEXITSTATUS 0 Nov 05 00:44:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_578(8161261) done Nov 05 00:44:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_584(8161267) WEXITSTATUS 0 Nov 05 00:44:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_584(8161267) done Nov 05 00:44:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_470(8160910) WEXITSTATUS 0 Nov 05 00:44:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_470(8160910) done Nov 05 00:45:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_7(8159188) WEXITSTATUS 0 Nov 05 00:45:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_7(8159188) done Nov 05 00:45:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_600(8161283) WEXITSTATUS 0 Nov 05 00:45:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_600(8161283) done Nov 05 00:45:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1278(8135233) WEXITSTATUS 0 Nov 05 00:45:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1278(8135233) done Nov 05 00:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1931(8151369) WEXITSTATUS 0 Nov 05 00:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1931(8151369) done Nov 05 00:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_442(8160882) WEXITSTATUS 0 Nov 05 00:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_442(8160882) done Nov 05 00:45:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_620(8161417) WEXITSTATUS 0 Nov 05 00:45:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_620(8161417) done Nov 05 00:45:24 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3884(8144949) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:45:24 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4617(8156196) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:45:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_6(8159187) WEXITSTATUS 0 Nov 05 00:45:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_6(8159187) done Nov 05 00:45:26 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3884(8144949) Nov 05 00:45:26 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4617(8156196) Nov 05 00:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_4(8159096) WEXITSTATUS 0 Nov 05 00:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_4(8159096) done Nov 05 00:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_618(8161415) WEXITSTATUS 0 Nov 05 00:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_618(8161415) done Nov 05 00:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_5(8159186) WEXITSTATUS 0 Nov 05 00:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_5(8159186) done Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:45:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5511(8160514) WEXITSTATUS 1 Nov 05 00:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5511(8160514) done Nov 05 00:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_516(8161073) WEXITSTATUS 0 Nov 05 00:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_516(8161073) done Nov 05 00:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5019(8158310) WEXITSTATUS 1 Nov 05 00:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5019(8158310) done Nov 05 00:45:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_464(8160904) WEXITSTATUS 0 Nov 05 00:45:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_464(8160904) done Nov 05 00:45:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_597(8161280) WEXITSTATUS 0 Nov 05 00:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_597(8161280) done Nov 05 00:46:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154605 WEXITSTATUS 0 Nov 05 00:46:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154605 done Nov 05 00:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_619(8161416) WEXITSTATUS 0 Nov 05 00:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_619(8161416) done Nov 05 00:46:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3575(8159162) WEXITSTATUS 0 Nov 05 00:46:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3575(8159162) done Nov 05 00:46:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_473(8160913) WEXITSTATUS 0 Nov 05 00:46:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_473(8160913) done Nov 05 00:46:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_576(8161259) WEXITSTATUS 0 Nov 05 00:46:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_576(8161259) done Nov 05 00:46:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_1(8159182) WEXITSTATUS 0 Nov 05 00:46:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159118_1(8159182) done Nov 05 00:46:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_610(8161293) WEXITSTATUS 0 Nov 05 00:46:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_610(8161293) done Nov 05 00:46:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_581(8161264) WEXITSTATUS 0 Nov 05 00:46:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_581(8161264) done Nov 05 00:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_629(8161426) WEXITSTATUS 0 Nov 05 00:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_629(8161426) done Nov 05 00:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_561(8161244) WEXITSTATUS 0 Nov 05 00:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_561(8161244) done Nov 05 00:46:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_5(8159097) WEXITSTATUS 0 Nov 05 00:46:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_5(8159097) done Nov 05 00:46:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:46:46 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:46:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:46:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 00:46:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_10(8159028) WEXITSTATUS 0 Nov 05 00:46:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_10(8159028) done Nov 05 00:46:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_579(8161262) WEXITSTATUS 0 Nov 05 00:46:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_579(8161262) done Nov 05 00:46:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5493(8160330) WEXITSTATUS 1 Nov 05 00:46:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5493(8160330) done Nov 05 00:46:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_612(8161409) WEXITSTATUS 0 Nov 05 00:46:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_612(8161409) done Nov 05 00:46:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_9(8159101) WEXITSTATUS 0 Nov 05 00:46:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_9(8159101) done Nov 05 00:46:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_585(8161268) WEXITSTATUS 0 Nov 05 00:46:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_585(8161268) done Nov 05 00:46:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_573(8161256) WEXITSTATUS 0 Nov 05 00:46:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_573(8161256) done Nov 05 00:46:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_3(8159095) WEXITSTATUS 0 Nov 05 00:46:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159028_3(8159095) done Nov 05 00:46:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5693(8162315) WEXITSTATUS 1 Nov 05 00:46:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5693(8162315) done Nov 05 00:47:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_602(8161285) WEXITSTATUS 0 Nov 05 00:47:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_602(8161285) done Nov 05 00:47:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_628(8161425) WEXITSTATUS 0 Nov 05 00:47:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_628(8161425) done Nov 05 00:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_563(8161246) WEXITSTATUS 0 Nov 05 00:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_563(8161246) done Nov 05 00:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_589(8161272) WEXITSTATUS 0 Nov 05 00:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_589(8161272) done Nov 05 00:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_522(8161079) WEXITSTATUS 0 Nov 05 00:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_522(8161079) done Nov 05 00:47:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_517(8161074) WEXITSTATUS 0 Nov 05 00:47:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_517(8161074) done Nov 05 00:47:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_502(8161059) WEXITSTATUS 0 Nov 05 00:47:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_502(8161059) done Nov 05 00:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_583(8161266) WEXITSTATUS 0 Nov 05 00:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_583(8161266) done Nov 05 00:47:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157533 WEXITSTATUS 0 Nov 05 00:47:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157533 done Nov 05 00:47:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1282(8135261) WEXITSTATUS 0 Nov 05 00:47:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1282(8135261) done Nov 05 00:47:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1291(8135290) WEXITSTATUS 0 Nov 05 00:47:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1291(8135290) done Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_573(8162483) in pi_ohern on r917u13n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_574(8162484) in pi_ohern on r917u13n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_575(8162485) in pi_ohern on r917u13n02 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_576(8162486) in pi_ohern on r917u13n02 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_577(8162487) in pi_ohern on r917u13n03 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_578(8162488) in pi_ohern on r917u13n03 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_579(8162489) in pi_ohern on r917u13n03 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_580(8162490) in pi_ohern on r917u13n04 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_581(8162491) in pi_ohern on r917u13n04 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_582(8162492) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_583(8162493) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_584(8162494) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_585(8162495) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_586(8162496) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_587(8162497) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_588(8162498) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_589(8162499) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_590(8162500) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_591(8162501) in pi_ohern on r917u29n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_592(8162502) in pi_ohern on r917u29n02 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_593(8162503) in pi_ohern on r917u29n02 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_594(8162504) in pi_ohern on r917u29n02 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_595(8162505) in pi_ohern on r917u29n02 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_596(8162506) in pi_ohern on r917u29n02 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_597(8162507) in pi_ohern on r917u29n02 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_598(8162508) in pi_ohern on r917u29n03 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_599(8162509) in pi_ohern on r917u29n03 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_600(8162510) in pi_ohern on r917u29n03 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_601(8162511) in pi_ohern on r917u29n04 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_602(8162512) in pi_ohern on r917u29n04 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_603(8162513) in pi_ohern on r807u24n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_604(8162514) in pi_ohern on r807u24n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_605(8162515) in pi_ohern on r903u26n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_606(8162516) in pi_ohern on r911u32n01 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_976(8128353) WEXITSTATUS 0 Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_976(8128353) done Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162190 NodeList=r816u35n10 #CPUs=12 Partition=scavenge Nov 05 00:47:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_607(8162517) in pi_ohern on r913u35n01 Nov 05 00:47:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159927 in day on r914u05n01 Nov 05 00:47:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159937 in day on r815u09n08 Nov 05 00:47:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159938 in day on r815u29n02 Nov 05 00:47:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159939 in day on r816u09n08 Nov 05 00:47:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8161677 in day on r816u03n08 Nov 05 00:47:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3638(8162518) in gpu on r903u34n01 Nov 05 00:47:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1881(8162519) in day on r815u29n03 Nov 05 00:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_635(8161432) WEXITSTATUS 0 Nov 05 00:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_635(8161432) done Nov 05 00:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_594(8161277) WEXITSTATUS 0 Nov 05 00:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_594(8161277) done Nov 05 00:47:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5718(8162520) in scavenge on r907u05n03 Nov 05 00:47:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5719(8162521) in scavenge on r916u04n01 Nov 05 00:47:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5720(8162522) in scavenge on r805u14n04 Nov 05 00:47:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5721(8162523) in scavenge on r918u32n04 Nov 05 00:47:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5722(8162524) in scavenge on r919u09n01 Nov 05 00:47:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5723(8162525) in scavenge on r919u09n01 Nov 05 00:47:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5724(8162526) in scavenge on r919u32n01 Nov 05 00:47:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5725(8162527) in scavenge on r815u09n06 Nov 05 00:47:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_471(8160911) WEXITSTATUS 0 Nov 05 00:47:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_471(8160911) done Nov 05 00:47:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_627(8161424) WEXITSTATUS 0 Nov 05 00:47:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_627(8161424) done Nov 05 00:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_637(8161434) WEXITSTATUS 0 Nov 05 00:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_637(8161434) done Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:47:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5504(8160341) WEXITSTATUS 1 Nov 05 00:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5504(8160341) done Nov 05 00:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5472(8159895) WEXITSTATUS 1 Nov 05 00:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5472(8159895) done Nov 05 00:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_559(8161242) WEXITSTATUS 0 Nov 05 00:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_559(8161242) done Nov 05 00:47:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_43(8156084) Nov 05 00:47:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_44(8156085) Nov 05 00:47:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_631(8161428) WEXITSTATUS 0 Nov 05 00:47:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_631(8161428) done Nov 05 00:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157537 WEXITSTATUS 0 Nov 05 00:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157537 done Nov 05 00:48:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_562(8161245) WEXITSTATUS 0 Nov 05 00:48:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_562(8161245) done Nov 05 00:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3625(8161710) WEXITSTATUS 0 Nov 05 00:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3625(8161710) done Nov 05 00:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_591(8161274) WEXITSTATUS 0 Nov 05 00:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_591(8161274) done Nov 05 00:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_577(8161260) WEXITSTATUS 0 Nov 05 00:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_577(8161260) done Nov 05 00:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_622(8161419) WEXITSTATUS 0 Nov 05 00:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_622(8161419) done Nov 05 00:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_554(8161237) WEXITSTATUS 0 Nov 05 00:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_554(8161237) done Nov 05 00:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5449(8159765) WEXITSTATUS 1 Nov 05 00:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5449(8159765) done Nov 05 00:48:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_615(8161412) WEXITSTATUS 0 Nov 05 00:48:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_615(8161412) done Nov 05 00:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133214 WEXITSTATUS 0 Nov 05 00:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133214 done Nov 05 00:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5636(8161838) WEXITSTATUS 1 Nov 05 00:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5636(8161838) done Nov 05 00:48:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_649(8161565) WEXITSTATUS 0 Nov 05 00:48:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_649(8161565) done Nov 05 00:48:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_638(8161435) WEXITSTATUS 0 Nov 05 00:48:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_638(8161435) done Nov 05 00:48:25 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157713 had to be killed Nov 05 00:48:25 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5185(8158539) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_595(8161278) WEXITSTATUS 0 Nov 05 00:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_595(8161278) done Nov 05 00:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_633(8161430) WEXITSTATUS 0 Nov 05 00:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_633(8161430) done Nov 05 00:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_642(8161439) WEXITSTATUS 0 Nov 05 00:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_642(8161439) done Nov 05 00:48:26 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5185(8158539) Nov 05 00:48:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4871(8157975) WEXITSTATUS 1 Nov 05 00:48:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4871(8157975) done Nov 05 00:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4668(8156330) WEXITSTATUS 1 Nov 05 00:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4668(8156330) done Nov 05 00:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1992(8125574) WEXITSTATUS 1 Nov 05 00:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1992(8125574) done Nov 05 00:48:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162317 NodeList=r815u23n07 #CPUs=12 Partition=scavenge Nov 05 00:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5233(8158636) WEXITSTATUS 1 Nov 05 00:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5233(8158636) done Nov 05 00:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5273(8158707) WEXITSTATUS 1 Nov 05 00:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5273(8158707) done Nov 05 00:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_590(8161273) WEXITSTATUS 0 Nov 05 00:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_590(8161273) done Nov 05 00:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_515(8161072) WEXITSTATUS 0 Nov 05 00:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_515(8161072) done Nov 05 00:48:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1294(8135300) WEXITSTATUS 0 Nov 05 00:48:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1294(8135300) done Nov 05 00:48:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_625(8161422) WEXITSTATUS 0 Nov 05 00:48:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_625(8161422) done Nov 05 00:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5338(8158957) WEXITSTATUS 1 Nov 05 00:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5338(8158957) done Nov 05 00:48:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4155(8150374) WEXITSTATUS 1 Nov 05 00:48:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4155(8150374) done Nov 05 00:48:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_588(8161271) WEXITSTATUS 0 Nov 05 00:48:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_588(8161271) done Nov 05 00:49:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8162528 NodeList=(null) usec=28652 Nov 05 00:49:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2263(8152180) WEXITSTATUS 0 Nov 05 00:49:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2263(8152180) done Nov 05 00:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162528 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 00:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1536(8146139) WEXITSTATUS 0 Nov 05 00:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1536(8146139) done Nov 05 00:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_521(8161078) WEXITSTATUS 0 Nov 05 00:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_521(8161078) done Nov 05 00:49:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157536 WEXITSTATUS 0 Nov 05 00:49:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157536 done Nov 05 00:49:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162318 NodeList=r816u15n11 #CPUs=12 Partition=scavenge Nov 05 00:49:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162529 InitPrio=32376 usec=5695 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131857 WEXITSTATUS 0 Nov 05 00:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131857 done Nov 05 00:49:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5371(8159181) WEXITSTATUS 1 Nov 05 00:49:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5371(8159181) done Nov 05 00:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_556(8161239) WEXITSTATUS 0 Nov 05 00:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_556(8161239) done Nov 05 00:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157534 WEXITSTATUS 0 Nov 05 00:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157534 done Nov 05 00:50:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162530 InitPrio=61916 usec=4039 Nov 05 00:50:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162531 InitPrio=32376 usec=4100 Nov 05 00:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1517(8143997) WEXITSTATUS 0 Nov 05 00:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1517(8143997) done Nov 05 00:50:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1303(8135327) WEXITSTATUS 0 Nov 05 00:50:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1303(8135327) done Nov 05 00:50:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_650(8161566) WEXITSTATUS 0 Nov 05 00:50:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_650(8161566) done Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_608(8162532) in pi_ohern on r917u13n01 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_609(8162533) in pi_ohern on r917u13n01 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_610(8162534) in pi_ohern on r917u13n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_611(8162535) in pi_ohern on r917u13n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_612(8162536) in pi_ohern on r917u13n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_613(8162537) in pi_ohern on r917u29n01 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_614(8162538) in pi_ohern on r917u29n01 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_615(8162539) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_616(8162540) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_617(8162541) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_618(8162542) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_619(8162543) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_620(8162544) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_621(8162545) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_622(8162546) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_623(8162547) in pi_ohern on r917u29n02 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_624(8162548) in pi_ohern on r917u29n03 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_625(8162549) in pi_ohern on r917u29n03 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_626(8162550) in pi_ohern on r917u29n03 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_627(8162551) in pi_ohern on r917u29n03 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_628(8162552) in pi_ohern on r917u29n03 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_629(8162553) in pi_ohern on r917u29n03 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_630(8162554) in pi_ohern on r917u29n04 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_631(8162555) in pi_ohern on r917u29n04 Nov 05 00:50:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_632(8162556) in pi_ohern on r917u29n04 Nov 05 00:50:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162530 in day on r904u07n[03-04] Nov 05 00:50:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159940 in day on r815u15n03 Nov 05 00:50:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3639(8162557) in gpu on r4513u29n01 Nov 05 00:50:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_634(8161431) WEXITSTATUS 0 Nov 05 00:50:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_634(8161431) done Nov 05 00:50:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1882(8162558) in day on r815u23n03 Nov 05 00:50:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133388 in gpu on r903u40n01 Nov 05 00:50:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n02 Nov 05 00:50:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5427(8159551) WEXITSTATUS 1 Nov 05 00:50:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5427(8159551) done Nov 05 00:50:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_615(7572784) Nov 05 00:50:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2076(8125873) Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5726(8162559) in scavenge on r802u06n03 Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5727(8162560) in scavenge on r912u05n04 Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5728(8162561) in scavenge on r916u01n01 Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5729(8162562) in scavenge on r917u01n02 Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5730(8162563) in scavenge on r815u23n08 Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5731(8162564) in scavenge on r919u09n03 Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5732(8162565) in scavenge on r815u15n07 Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5733(8162566) in scavenge on r815u29n01 Nov 05 00:50:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5734(8162567) in scavenge on r816u03n01 Nov 05 00:50:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162568 InitPrio=32376 usec=3920 Nov 05 00:50:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3637(8162479) WEXITSTATUS 0 Nov 05 00:50:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3637(8162479) done Nov 05 00:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_624(8161421) WEXITSTATUS 0 Nov 05 00:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_624(8161421) done Nov 05 00:50:54 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_45(8156099) Nov 05 00:50:54 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157670 had to be killed Nov 05 00:50:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_643(8161440) WEXITSTATUS 0 Nov 05 00:50:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_643(8161440) done Nov 05 00:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_652(8161568) WEXITSTATUS 0 Nov 05 00:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_652(8161568) done Nov 05 00:51:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162569 InitPrio=32376 usec=5368 Nov 05 00:51:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131870 WEXITSTATUS 0 Nov 05 00:51:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131870 done Nov 05 00:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_621(8161418) WEXITSTATUS 0 Nov 05 00:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_621(8161418) done Nov 05 00:51:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_648(8161564) WEXITSTATUS 0 Nov 05 00:51:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_648(8161564) done Nov 05 00:51:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3639(8162557) WEXITSTATUS 0 Nov 05 00:51:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3639(8162557) done Nov 05 00:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3636(8162478) WEXITSTATUS 0 Nov 05 00:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3636(8162478) done Nov 05 00:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_639(8161436) WEXITSTATUS 0 Nov 05 00:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_639(8161436) done Nov 05 00:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3638(8162518) WEXITSTATUS 0 Nov 05 00:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3638(8162518) done Nov 05 00:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_587(8161270) WEXITSTATUS 0 Nov 05 00:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_587(8161270) done Nov 05 00:51:25 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_632(8161429) WEXITSTATUS 0 Nov 05 00:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_632(8161429) done Nov 05 00:51:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133217 WEXITSTATUS 0 Nov 05 00:51:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133217 done Nov 05 00:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5356(8159090) WEXITSTATUS 1 Nov 05 00:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5356(8159090) done Nov 05 00:51:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5123(8158463) WEXITSTATUS 1 Nov 05 00:51:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5123(8158463) done Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:51:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:51:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5708(8162421) WEXITSTATUS 1 Nov 05 00:51:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5708(8162421) done Nov 05 00:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_586(8161269) WEXITSTATUS 0 Nov 05 00:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_586(8161269) done Nov 05 00:52:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_592(8161275) WEXITSTATUS 0 Nov 05 00:52:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_592(8161275) done Nov 05 00:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5721(8162523) WEXITSTATUS 1 Nov 05 00:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5721(8162523) done Nov 05 00:52:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_647(8161563) WEXITSTATUS 0 Nov 05 00:52:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_647(8161563) done Nov 05 00:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_641(8161438) WEXITSTATUS 0 Nov 05 00:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_641(8161438) done Nov 05 00:52:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5431(8159555) WEXITSTATUS 1 Nov 05 00:52:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5431(8159555) done Nov 05 00:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_653(8161679) WEXITSTATUS 0 Nov 05 00:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_653(8161679) done Nov 05 00:52:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133322 WEXITSTATUS 0 Nov 05 00:52:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133322 done Nov 05 00:52:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2249(8152166) WEXITSTATUS 0 Nov 05 00:52:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2249(8152166) done Nov 05 00:52:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1539(8146142) WEXITSTATUS 0 Nov 05 00:52:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1539(8146142) done Nov 05 00:52:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_630(8161427) WEXITSTATUS 0 Nov 05 00:52:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_630(8161427) done Nov 05 00:52:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5416(8159461) WEXITSTATUS 1 Nov 05 00:52:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5416(8159461) done Nov 05 00:52:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159662 WEXITSTATUS 0 Nov 05 00:52:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159662 done Nov 05 00:52:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5473(8159896) WEXITSTATUS 1 Nov 05 00:52:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5473(8159896) done Nov 05 00:52:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_651(8161567) WEXITSTATUS 0 Nov 05 00:52:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_651(8161567) done Nov 05 00:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_520(8161077) WEXITSTATUS 0 Nov 05 00:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_520(8161077) done Nov 05 00:52:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2078(8125880) Nov 05 00:53:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5170(8158524) WEXITSTATUS 1 Nov 05 00:53:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5170(8158524) done Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_633(8162570) in pi_ohern on r917u29n01 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_634(8162571) in pi_ohern on r917u29n01 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_635(8162572) in pi_ohern on r917u29n01 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_636(8162573) in pi_ohern on r917u29n02 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_637(8162574) in pi_ohern on r917u29n02 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_638(8162575) in pi_ohern on r917u29n02 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_639(8162576) in pi_ohern on r917u29n02 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_640(8162577) in pi_ohern on r917u29n02 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_641(8162578) in pi_ohern on r917u29n02 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_642(8162579) in pi_ohern on r917u29n02 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_643(8162580) in pi_ohern on r917u29n03 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_644(8162581) in pi_ohern on r917u29n03 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_645(8162582) in pi_ohern on r917u29n03 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_646(8162583) in pi_ohern on r917u29n03 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_647(8162584) in pi_ohern on r917u29n04 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_648(8162585) in pi_ohern on r917u29n04 Nov 05 00:53:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_649(8162586) in pi_ohern on r917u29n04 Nov 05 00:53:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162529 in day on r815u35n03 Nov 05 00:53:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3640(8162587) in gpu on r903u34n01 Nov 05 00:53:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3641(8162588) in gpu on r808u08n01 Nov 05 00:53:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3642(8162589) in gpu on r808u17n01 Nov 05 00:53:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3643(8162590) in gpu on r4513u29n01 Nov 05 00:53:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133389 in gpu on r806u35n01 Nov 05 00:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5735(8162591) in scavenge on r915u35n04 Nov 05 00:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5736(8162592) in scavenge on r802u04n03 Nov 05 00:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5737(8162593) in scavenge on r802u09n03 Nov 05 00:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5738(8162594) in scavenge on r805u27n04 Nov 05 00:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5739(8162595) in scavenge on r806u02n03 Nov 05 00:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5740(8162596) in scavenge on r913u13n03 Nov 05 00:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5741(8162597) in scavenge on r918u32n04 Nov 05 00:53:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157538 WEXITSTATUS 0 Nov 05 00:53:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157538 done Nov 05 00:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1261(8135166) WEXITSTATUS 0 Nov 05 00:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1261(8135166) done Nov 05 00:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_636(8161433) WEXITSTATUS 0 Nov 05 00:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_636(8161433) done Nov 05 00:53:25 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 00:53:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5600(8161687) WEXITSTATUS 1 Nov 05 00:53:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5600(8161687) done Nov 05 00:53:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1523(8144451) WEXITSTATUS 0 Nov 05 00:53:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1523(8144451) done Nov 05 00:53:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5298(8158760) WEXITSTATUS 1 Nov 05 00:53:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5298(8158760) done Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159909 WEXITSTATUS 0 Nov 05 00:53:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159909 done Nov 05 00:53:54 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5556(8161046) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:53:54 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5575(8161229) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4478(8155851) WEXITSTATUS 1 Nov 05 00:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4478(8155851) done Nov 05 00:53:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5556(8161046) Nov 05 00:53:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5575(8161229) Nov 05 00:54:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5716(8162481) WEXITSTATUS 1 Nov 05 00:54:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5716(8162481) done Nov 05 00:54:13 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r902u10n01 reason set to: NHC: check_cmd_status: 142 (expected 0) returned by "test -e /vast/palmer/home.grace/.yalehpc". Nov 05 00:54:13 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r902u10n01 state set to DRAINING Nov 05 00:54:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 00:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_626(8161423) WEXITSTATUS 0 Nov 05 00:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_626(8161423) done Nov 05 00:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_640(8161437) WEXITSTATUS 0 Nov 05 00:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_640(8161437) done Nov 05 00:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2381(8152425) WEXITSTATUS 0 Nov 05 00:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2381(8152425) done Nov 05 00:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1451(8136671) WEXITSTATUS 0 Nov 05 00:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1451(8136671) done Nov 05 00:54:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_623(8161420) WEXITSTATUS 0 Nov 05 00:54:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158906_623(8161420) done Nov 05 00:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4206(8152044) WEXITSTATUS 1 Nov 05 00:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4206(8152044) done Nov 05 00:55:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157539 WEXITSTATUS 0 Nov 05 00:55:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157539 done Nov 05 00:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 00:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 00:55:25 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 00:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154619 WEXITSTATUS 0 Nov 05 00:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154619 done Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:55:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:55:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_650(8162598) in pi_ohern on r917u29n01 Nov 05 00:55:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_651(8162599) in pi_ohern on r917u29n02 Nov 05 00:55:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_652(8162600) in pi_ohern on r917u29n02 Nov 05 00:55:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_653(8162601) in pi_ohern on r917u29n03 Nov 05 00:55:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162531 in day on r815u23n04 Nov 05 00:55:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162568 in day on r815u23n04 Nov 05 00:55:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162569 in day on r815u03n04 Nov 05 00:55:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1883(8162602) in day on r816u03n07 Nov 05 00:55:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n02 Nov 05 00:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_651(8162599) WEXITSTATUS 0 Nov 05 00:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_651(8162599) done Nov 05 00:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_652(8162600) WEXITSTATUS 0 Nov 05 00:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_652(8162600) done Nov 05 00:56:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_653(8162601) WEXITSTATUS 0 Nov 05 00:56:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_653(8162601) done Nov 05 00:56:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5742(8162603) in scavenge on r801u09n03 Nov 05 00:56:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5743(8162604) in scavenge on r802u02n02 Nov 05 00:56:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5744(8162605) in scavenge on r903u07n01 Nov 05 00:56:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5745(8162606) in scavenge on r816u29n10 Nov 05 00:56:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5746(8162607) in scavenge on r816u29n11 Nov 05 00:56:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5747(8162608) in scavenge on r903u34n01 Nov 05 00:56:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5748(8162609) in scavenge on r904u04n01 Nov 05 00:56:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5749(8162610) in scavenge on r919u09n01 Nov 05 00:56:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1458(8137918) in scavenge on r816u29n11 Nov 05 00:56:11 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 00:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1479(8138593) WEXITSTATUS 0 Nov 05 00:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1479(8138593) done Nov 05 00:56:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5576(8161230) WEXITSTATUS 1 Nov 05 00:56:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5576(8161230) done Nov 05 00:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1296(8135309) WEXITSTATUS 0 Nov 05 00:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1296(8135309) done Nov 05 00:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5718(8162520) WEXITSTATUS 1 Nov 05 00:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5718(8162520) done Nov 05 00:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1515(8143788) WEXITSTATUS 0 Nov 05 00:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1515(8143788) done Nov 05 00:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5409(8159390) WEXITSTATUS 1 Nov 05 00:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5409(8159390) done Nov 05 00:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161845 WEXITSTATUS 127 Nov 05 00:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161845 done Nov 05 00:56:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159906 WEXITSTATUS 0 Nov 05 00:56:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159906 done Nov 05 00:56:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5448(8159764) WEXITSTATUS 1 Nov 05 00:56:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5448(8159764) done Nov 05 00:57:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133321 WEXITSTATUS 0 Nov 05 00:57:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133321 done Nov 05 00:57:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8161051 Nov 05 00:57:27 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 00:57:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5723(8162525) WEXITSTATUS 1 Nov 05 00:57:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5723(8162525) done Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:57:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5743(8162604) WEXITSTATUS 1 Nov 05 00:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5743(8162604) done Nov 05 00:57:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_47(8156184) Nov 05 00:58:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159907 WEXITSTATUS 0 Nov 05 00:58:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159907 done Nov 05 00:58:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5202(8158565) WEXITSTATUS 1 Nov 05 00:58:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5202(8158565) done Nov 05 00:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1275(8135230) WEXITSTATUS 0 Nov 05 00:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1275(8135230) done Nov 05 00:58:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8154980 Nov 05 00:58:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_654(8162611) in pi_ohern on r917u29n02 Nov 05 00:58:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_655(8162612) in pi_ohern on r917u29n02 Nov 05 00:58:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_656(8162613) in pi_ohern on r917u29n03 Nov 05 00:58:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162614 InitPrio=21238 usec=7433 Nov 05 00:58:43 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 00:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907377 WEXITSTATUS 0 Nov 05 00:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907377 done Nov 05 00:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_655(8162612) WEXITSTATUS 0 Nov 05 00:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_655(8162612) done Nov 05 00:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5405(8159386) WEXITSTATUS 1 Nov 05 00:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5405(8159386) done Nov 05 00:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_654(8162611) WEXITSTATUS 0 Nov 05 00:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_654(8162611) done Nov 05 00:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_656(8162613) WEXITSTATUS 0 Nov 05 00:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_656(8162613) done Nov 05 00:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5750(8162615) in scavenge on r907u05n03 Nov 05 00:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5751(8162616) in scavenge on r801u06n01 Nov 05 00:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5752(8162617) in scavenge on r802u02n02 Nov 05 00:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5753(8162618) in scavenge on r802u04n04 Nov 05 00:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5754(8162619) in scavenge on r919u01n04 Nov 05 00:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5755(8162620) in scavenge on r919u05n04 Nov 05 00:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5756(8162621) in scavenge on r919u09n01 Nov 05 00:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5757(8162622) in scavenge on r816u35n05 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 00:58:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 00:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154622 WEXITSTATUS 0 Nov 05 00:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154622 done Nov 05 00:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4750(8157100) WEXITSTATUS 1 Nov 05 00:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4750(8157100) done Nov 05 00:59:11 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r902u10n01 state set to ALLOCATED Nov 05 00:59:12 r912u23n01 slurmctld[163760]: slurmctld: Node r902u10n01 now responding Nov 05 00:59:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 00:59:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162623 InitPrio=34888 usec=3482 Nov 05 00:59:25 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4752(8157102) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:59:25 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157723 had to be killed Nov 05 00:59:26 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4752(8157102) Nov 05 00:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5695(8162357) WEXITSTATUS 1 Nov 05 00:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5695(8162357) done Nov 05 00:59:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4917(8158080) WEXITSTATUS 1 Nov 05 00:59:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4917(8158080) done Nov 05 00:59:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3640(8162587) WEXITSTATUS 0 Nov 05 00:59:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3640(8162587) done Nov 05 00:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5234(8158637) WEXITSTATUS 1 Nov 05 00:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5234(8158637) done Nov 05 00:59:56 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4924(8158111) has been requeued to reclaim resources for JobId=Invalid Nov 05 00:59:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4924(8158111) Nov 05 00:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162623 NodeList=r801u06n02 #CPUs=4 Partition=education Nov 05 01:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 01:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 01:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 01:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 01:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 01:00:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5538(8160873) WEXITSTATUS 1 Nov 05 01:00:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5538(8160873) done Nov 05 01:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5076(8158388) WEXITSTATUS 1 Nov 05 01:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5076(8158388) done Nov 05 01:00:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5367(8159177) WEXITSTATUS 1 Nov 05 01:00:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5367(8159177) done Nov 05 01:00:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1417(8135950) WEXITSTATUS 0 Nov 05 01:00:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1417(8135950) done Nov 05 01:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159918 WEXITSTATUS 0 Nov 05 01:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159918 done Nov 05 01:00:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154609 WEXITSTATUS 0 Nov 05 01:00:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154609 done Nov 05 01:00:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5596(8161672) WEXITSTATUS 1 Nov 05 01:00:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5596(8161672) done Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:00:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:01:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8155209 Nov 05 01:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_657(8162624) in pi_ohern on r917u29n02 Nov 05 01:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_658(8162625) in pi_ohern on r917u29n02 Nov 05 01:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_659(8162626) in pi_ohern on r917u29n03 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5732(8162565) WEXITSTATUS 1 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5732(8162565) done Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5480(8160129) WEXITSTATUS 1 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5480(8160129) done Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r904u04n02 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r904u04n02 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159929 in day on r914u01n03 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907378 in gpu on r805u38n01 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3644(8162627) in gpu on r903u34n01 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154658 in day on r815u35n06 Nov 05 01:01:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154659 in day on r816u09n07 Nov 05 01:01:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_592(7572761) WEXITSTATUS 0 Nov 05 01:01:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_592(7572761) done Nov 05 01:01:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_885(7573054) WEXITSTATUS 0 Nov 05 01:01:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_885(7573054) done Nov 05 01:01:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n02 Nov 05 01:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_657(8162624) WEXITSTATUS 0 Nov 05 01:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_657(8162624) done Nov 05 01:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_658(8162625) WEXITSTATUS 0 Nov 05 01:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_658(8162625) done Nov 05 01:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_659(8162626) WEXITSTATUS 0 Nov 05 01:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_659(8162626) done Nov 05 01:01:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5758(8162628) in scavenge on r907u05n02 Nov 05 01:01:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5759(8162629) in scavenge on r802u11n03 Nov 05 01:01:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5760(8162630) in scavenge on r917u01n04 Nov 05 01:01:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5761(8162631) in scavenge on r907u28n01 Nov 05 01:01:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5762(8162632) in scavenge on r907u34n01 Nov 05 01:01:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5763(8162633) in scavenge on r912u28n02 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162634 InitPrio=32376 usec=3694 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162635 InitPrio=32376 usec=3450 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162636 InitPrio=32376 usec=4765 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162637 InitPrio=32376 usec=3869 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162638 InitPrio=32376 usec=4147 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162639 InitPrio=32376 usec=2427 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162640 InitPrio=32376 usec=2169 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162641 InitPrio=32376 usec=2096 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162642 InitPrio=32376 usec=3342 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162643 InitPrio=32376 usec=2847 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162644 InitPrio=32376 usec=2564 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162645 InitPrio=32376 usec=2783 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162646 InitPrio=32376 usec=2385 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162647 InitPrio=32376 usec=2482 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162648 InitPrio=32376 usec=3550 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162649 InitPrio=32376 usec=2763 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162650 InitPrio=32376 usec=2292 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162651 InitPrio=32376 usec=2098 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162652 InitPrio=32376 usec=2490 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162653 InitPrio=32376 usec=2388 Nov 05 01:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162654 InitPrio=32376 usec=2401 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162655 InitPrio=32376 usec=5861 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162656 InitPrio=32376 usec=4203 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162657 InitPrio=32376 usec=3142 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162658 InitPrio=32376 usec=2298 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162659 InitPrio=32376 usec=3970 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162660 InitPrio=32376 usec=3226 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162661 InitPrio=32376 usec=2479 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162662 InitPrio=32376 usec=2945 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162663 InitPrio=32376 usec=2357 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162664 InitPrio=32376 usec=2841 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162665 InitPrio=32376 usec=2584 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162666 InitPrio=32376 usec=2281 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162667 InitPrio=32376 usec=1994 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162668 InitPrio=32376 usec=2019 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162669 InitPrio=32376 usec=2872 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162670 InitPrio=32376 usec=2426 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162671 InitPrio=32376 usec=3285 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162672 InitPrio=32376 usec=2781 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162673 InitPrio=32376 usec=2315 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162674 InitPrio=32376 usec=2203 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162675 InitPrio=32376 usec=2060 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162676 InitPrio=32376 usec=2517 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162677 InitPrio=32376 usec=2337 Nov 05 01:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162678 InitPrio=32376 usec=2267 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162679 InitPrio=32376 usec=4027 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162680 InitPrio=32376 usec=4116 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162681 InitPrio=32376 usec=3260 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162682 InitPrio=32376 usec=2200 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162683 InitPrio=32376 usec=2109 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162684 InitPrio=32376 usec=3968 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162685 InitPrio=32376 usec=2857 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162686 InitPrio=32376 usec=2187 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162687 InitPrio=32376 usec=2727 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162688 InitPrio=32376 usec=2387 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162689 InitPrio=32376 usec=2064 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162690 InitPrio=32376 usec=2573 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162691 InitPrio=32376 usec=2119 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162692 InitPrio=32376 usec=1982 Nov 05 01:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162693 InitPrio=32376 usec=1953 Nov 05 01:01:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5490(8160139) WEXITSTATUS 1 Nov 05 01:01:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5490(8160139) done Nov 05 01:01:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5260(8158694) WEXITSTATUS 1 Nov 05 01:01:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5260(8158694) done Nov 05 01:01:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159910 WEXITSTATUS 0 Nov 05 01:01:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159910 done Nov 05 01:01:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 01:01:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 01:01:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 01:01:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5572(8161226) WEXITSTATUS 1 Nov 05 01:01:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5572(8161226) done Nov 05 01:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154601 WEXITSTATUS 0 Nov 05 01:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154601 done Nov 05 01:02:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5354(8159088) WEXITSTATUS 1 Nov 05 01:02:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5354(8159088) done Nov 05 01:02:25 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157546 had to be killed Nov 05 01:02:25 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5545(8160880) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:02:27 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5545(8160880) Nov 05 01:02:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162694 InitPrio=17550 usec=6148 Nov 05 01:02:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162694 NodeList=r801u09n03 #CPUs=12 Partition=pi_esi Nov 05 01:02:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:02:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:03:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162695 InitPrio=17550 usec=4818 Nov 05 01:03:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162696 InitPrio=17550 usec=3724 Nov 05 01:03:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5101(8158434) WEXITSTATUS 1 Nov 05 01:03:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5101(8158434) done Nov 05 01:03:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2296(8152213) WEXITSTATUS 0 Nov 05 01:03:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2296(8152213) done Nov 05 01:03:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3575(8136197) WEXITSTATUS 1 Nov 05 01:03:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3575(8136197) done Nov 05 01:03:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_3(8157551) WEXITSTATUS 0 Nov 05 01:03:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157465_3(8157551) done Nov 05 01:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5664(8162031) WEXITSTATUS 1 Nov 05 01:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5664(8162031) done Nov 05 01:04:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_660(8162697) in pi_ohern on r917u29n02 Nov 05 01:04:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_661(8162698) in pi_ohern on r917u29n02 Nov 05 01:04:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_662(8162699) in pi_ohern on r917u29n03 Nov 05 01:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159936 in day on r909u07n03 Nov 05 01:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162636 in day on r816u03n07 Nov 05 01:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162637 in day on r815u23n05 Nov 05 01:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162638 in day on r815u35n06 Nov 05 01:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162639 in day on r815u03n03 Nov 05 01:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162640 in day on r815u09n03 Nov 05 01:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162641 in day on r815u09n05 Nov 05 01:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162642 in day on r815u15n08 Nov 05 01:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162614 in day on r918u05n01,r918u36n01 Nov 05 01:04:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_662(8162699) WEXITSTATUS 0 Nov 05 01:04:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_662(8162699) done Nov 05 01:04:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_661(8162698) WEXITSTATUS 0 Nov 05 01:04:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_661(8162698) done Nov 05 01:04:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_660(8162697) WEXITSTATUS 0 Nov 05 01:04:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_660(8162697) done Nov 05 01:04:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4710(8156593) WEXITSTATUS 1 Nov 05 01:04:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4710(8156593) done Nov 05 01:04:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149757_49(8149757) Nov 05 01:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5764(8162700) in scavenge on r801u09n04 Nov 05 01:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5765(8162701) in scavenge on r903u01n01 Nov 05 01:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5766(8162702) in scavenge on r802u02n03 Nov 05 01:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5767(8162703) in scavenge on r802u09n03 Nov 05 01:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5768(8162704) in scavenge on r816u03n07 Nov 05 01:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5769(8162705) in scavenge on r816u29n05 Nov 05 01:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5770(8162706) in scavenge on r904u13n01 Nov 05 01:04:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4065(8147918) WEXITSTATUS 1 Nov 05 01:04:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4065(8147918) done Nov 05 01:04:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162707 InitPrio=61916 usec=7378 Nov 05 01:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5548(8161038) WEXITSTATUS 1 Nov 05 01:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5548(8161038) done Nov 05 01:04:55 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3929(8145653) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:04:55 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157712 had to be killed Nov 05 01:04:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3929(8145653) Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:04:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:05:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159908 WEXITSTATUS 0 Nov 05 01:05:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159908 done Nov 05 01:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5501(8160338) WEXITSTATUS 1 Nov 05 01:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5501(8160338) done Nov 05 01:05:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8155340 Nov 05 01:05:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162707 NodeList=r904u10n02,r914u01n[02-03] #CPUs=4 Partition=day Nov 05 01:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5735(8162591) WEXITSTATUS 1 Nov 05 01:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5735(8162591) done Nov 05 01:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5325(8158898) WEXITSTATUS 1 Nov 05 01:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5325(8158898) done Nov 05 01:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3608(8160328) WEXITSTATUS 0 Nov 05 01:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3608(8160328) done Nov 05 01:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5256(8158680) WEXITSTATUS 1 Nov 05 01:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5256(8158680) done Nov 05 01:05:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2117(8125928) Nov 05 01:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154626 WEXITSTATUS 0 Nov 05 01:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154626 done Nov 05 01:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1467(8138362) WEXITSTATUS 0 Nov 05 01:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1467(8138362) done Nov 05 01:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4449(8155762) WEXITSTATUS 1 Nov 05 01:06:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4449(8155762) done Nov 05 01:06:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3642(8162589) WEXITSTATUS 0 Nov 05 01:06:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3642(8162589) done Nov 05 01:06:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_5(8156214) Nov 05 01:06:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154610 WEXITSTATUS 0 Nov 05 01:06:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154610 done Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:06:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:07:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_663(8162708) in pi_ohern on r917u29n02 Nov 05 01:07:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_664(8162709) in pi_ohern on r917u29n02 Nov 05 01:07:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_665(8162710) in pi_ohern on r917u29n03 Nov 05 01:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162643 in day on r815u23n03 Nov 05 01:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162644 in day on r815u35n04 Nov 05 01:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162645 in day on r815u09n03 Nov 05 01:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162646 in day on r815u09n06 Nov 05 01:07:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162647 in day on r815u09n07 Nov 05 01:07:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3645(8162711) in gpu on r808u17n01 Nov 05 01:07:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3646(8162712) in gpu on r4513u23n01 Nov 05 01:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_664(8162709) WEXITSTATUS 0 Nov 05 01:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_664(8162709) done Nov 05 01:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_665(8162710) WEXITSTATUS 0 Nov 05 01:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_665(8162710) done Nov 05 01:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_663(8162708) WEXITSTATUS 0 Nov 05 01:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_663(8162708) done Nov 05 01:07:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 01:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2388(8152432) WEXITSTATUS 0 Nov 05 01:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2388(8152432) done Nov 05 01:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5771(8162713) in scavenge on r801u09n04 Nov 05 01:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5772(8162714) in scavenge on r801u23n01 Nov 05 01:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5773(8162715) in scavenge on r915u35n04 Nov 05 01:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5774(8162716) in scavenge on r917u01n04 Nov 05 01:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5775(8162717) in scavenge on r816u29n10 Nov 05 01:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5776(8162718) in scavenge on r805u09n03 Nov 05 01:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5777(8162719) in scavenge on r805u34n01 Nov 05 01:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159911 WEXITSTATUS 0 Nov 05 01:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159911 done Nov 05 01:07:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_6(8156215) Nov 05 01:07:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1449(8136424) WEXITSTATUS 0 Nov 05 01:07:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1449(8136424) done Nov 05 01:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157542 WEXITSTATUS 0 Nov 05 01:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157542 done Nov 05 01:07:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157540 WEXITSTATUS 0 Nov 05 01:07:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157540 done Nov 05 01:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154633 WEXITSTATUS 0 Nov 05 01:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154633 done Nov 05 01:07:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162720 InitPrio=61916 usec=4221 Nov 05 01:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131869 WEXITSTATUS 0 Nov 05 01:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131869 done Nov 05 01:07:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159915 WEXITSTATUS 0 Nov 05 01:07:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159915 done Nov 05 01:07:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_7(8156232) Nov 05 01:07:55 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157706 had to be killed Nov 05 01:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131874 WEXITSTATUS 0 Nov 05 01:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131874 done Nov 05 01:08:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4854(8157879) WEXITSTATUS 1 Nov 05 01:08:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4854(8157879) done Nov 05 01:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5430(8159554) WEXITSTATUS 1 Nov 05 01:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5430(8159554) done Nov 05 01:08:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154623 WEXITSTATUS 0 Nov 05 01:08:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154623 done Nov 05 01:08:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4585(8156098) WEXITSTATUS 1 Nov 05 01:08:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4585(8156098) done Nov 05 01:08:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_891(7573060) Nov 05 01:08:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2120(8125932) Nov 05 01:08:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157541 WEXITSTATUS 0 Nov 05 01:08:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157541 done Nov 05 01:08:26 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4212(8152267) WEXITSTATUS 1 Nov 05 01:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4212(8152267) done Nov 05 01:08:31 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162720 NodeList=r806u02n[02,04] #CPUs=4 Partition=day Nov 05 01:08:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133213 WEXITSTATUS 0 Nov 05 01:08:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133213 done Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:08:59 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:09:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3626(8161835) WEXITSTATUS 0 Nov 05 01:09:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3626(8161835) done Nov 05 01:09:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154620 WEXITSTATUS 0 Nov 05 01:09:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154620 done Nov 05 01:09:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8162720 uid 24447 Nov 05 01:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1350(8135503) WEXITSTATUS 0 Nov 05 01:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1350(8135503) done Nov 05 01:09:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162721 InitPrio=61916 usec=4326 Nov 05 01:09:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1329(8135426) WEXITSTATUS 0 Nov 05 01:09:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1329(8135426) done Nov 05 01:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1457(8137386) WEXITSTATUS 0 Nov 05 01:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1457(8137386) done Nov 05 01:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_64(8160087) WEXITSTATUS 0 Nov 05 01:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_64(8160087) done Nov 05 01:09:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5495(8160332) WEXITSTATUS 1 Nov 05 01:09:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5495(8160332) done Nov 05 01:09:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_666(8162722) in pi_ohern on r917u29n02 Nov 05 01:09:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_667(8162723) in pi_ohern on r917u29n02 Nov 05 01:09:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159029_668(8159029) in pi_ohern on r917u29n03 Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157544 WEXITSTATUS 0 Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157544 done Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162721 in day on r806u02n[02,04] Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162634 in day on r918u36n03 Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162649 in day on r815u35n05 Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162650 in day on r816u03n06 Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162651 in day on r815u03n02 Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162652 in day on r815u09n04 Nov 05 01:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162653 in day on r815u15n02 Nov 05 01:09:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3647(8162724) in gpu on r808u14n01 Nov 05 01:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_66(8160089) WEXITSTATUS 0 Nov 05 01:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_66(8160089) done Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_668(8159029) WEXITSTATUS 0 Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_668(8159029) done Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_666(8162722) WEXITSTATUS 0 Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_666(8162722) done Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_667(8162723) WEXITSTATUS 0 Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_667(8162723) done Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1884(8162725) in day on r815u09n05 Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1885(8162726) in day on r815u23n04 Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1886(8162727) in day on r816u09n06 Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133390 in gpu on r806u35n01 Nov 05 01:09:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133391 in gpu on r806u35n01 Nov 05 01:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_69(8160092) WEXITSTATUS 0 Nov 05 01:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_69(8160092) done Nov 05 01:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3635(8162477) WEXITSTATUS 0 Nov 05 01:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3635(8162477) done Nov 05 01:10:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5778(8162728) in scavenge on r802u11n01 Nov 05 01:10:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5779(8162729) in scavenge on r802u32n04 Nov 05 01:10:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5780(8162730) in scavenge on r816u15n11 Nov 05 01:10:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5781(8162731) in scavenge on r816u23n12 Nov 05 01:10:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5782(8162732) in scavenge on r808u08n01 Nov 05 01:10:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5783(8162733) in scavenge on r815u29n04 Nov 05 01:10:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5784(8162734) in scavenge on r815u29n04 Nov 05 01:10:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5785(8162735) in scavenge on r815u29n04 Nov 05 01:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_68(8160091) WEXITSTATUS 0 Nov 05 01:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_68(8160091) done Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:10:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:10:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_65(8160088) WEXITSTATUS 0 Nov 05 01:10:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_65(8160088) done Nov 05 01:10:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157543 WEXITSTATUS 0 Nov 05 01:10:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157543 done Nov 05 01:10:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154634 WEXITSTATUS 0 Nov 05 01:10:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154634 done Nov 05 01:10:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_61(8160084) WEXITSTATUS 0 Nov 05 01:10:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_61(8160084) done Nov 05 01:10:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 01:10:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 01:10:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 01:10:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1925(8151363) WEXITSTATUS 0 Nov 05 01:10:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1925(8151363) done Nov 05 01:10:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4736(8156976) WEXITSTATUS 1 Nov 05 01:10:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4736(8156976) done Nov 05 01:10:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4665(8156327) WEXITSTATUS 1 Nov 05 01:10:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4665(8156327) done Nov 05 01:10:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8162736 NodeList=(null) usec=30090 Nov 05 01:10:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5540(8160875) WEXITSTATUS 1 Nov 05 01:10:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5540(8160875) done Nov 05 01:11:00 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162736 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 01:11:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_63(8160086) WEXITSTATUS 0 Nov 05 01:11:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_63(8160086) done Nov 05 01:11:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131873 WEXITSTATUS 0 Nov 05 01:11:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131873 done Nov 05 01:11:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4277(8153940) WEXITSTATUS 1 Nov 05 01:11:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4277(8153940) done Nov 05 01:11:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5632(8161752) WEXITSTATUS 1 Nov 05 01:11:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5632(8161752) done Nov 05 01:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5142(8158489) WEXITSTATUS 1 Nov 05 01:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5142(8158489) done Nov 05 01:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5756(8162621) WEXITSTATUS 1 Nov 05 01:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5756(8162621) done Nov 05 01:12:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_60(8160083) WEXITSTATUS 0 Nov 05 01:12:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_60(8160083) done Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:12:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:12:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1273(8135222) WEXITSTATUS 0 Nov 05 01:12:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1273(8135222) done Nov 05 01:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157547 WEXITSTATUS 0 Nov 05 01:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157547 done Nov 05 01:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_62(8160085) WEXITSTATUS 0 Nov 05 01:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_62(8160085) done Nov 05 01:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1568(8149613) WEXITSTATUS 0 Nov 05 01:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1568(8149613) done Nov 05 01:12:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4399(8155329) WEXITSTATUS 1 Nov 05 01:12:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4399(8155329) done Nov 05 01:12:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4901(8158034) WEXITSTATUS 1 Nov 05 01:12:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4901(8158034) done Nov 05 01:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_1(8162737) in pi_ohern on r917u29n02 Nov 05 01:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_2(8162738) in pi_ohern on r917u29n02 Nov 05 01:12:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_3(8162739) in pi_ohern on r917u29n03 Nov 05 01:12:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162635 in day on r913u28n02 Nov 05 01:12:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3648(8162740) in gpu on r903u40n01 Nov 05 01:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_59(8160082) WEXITSTATUS 0 Nov 05 01:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_59(8160082) done Nov 05 01:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_67(8160090) WEXITSTATUS 0 Nov 05 01:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_67(8160090) done Nov 05 01:12:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133392 in gpu on r806u37n01 Nov 05 01:12:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 01:12:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5433(8159557) WEXITSTATUS 1 Nov 05 01:12:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5433(8159557) done Nov 05 01:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_57(8160080) WEXITSTATUS 0 Nov 05 01:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_57(8160080) done Nov 05 01:12:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5786(8162741) in scavenge on r802u04n04 Nov 05 01:12:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5787(8162742) in scavenge on r816u35n09 Nov 05 01:12:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5788(8162743) in scavenge on r904u04n02 Nov 05 01:12:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5789(8162744) in scavenge on r904u10n03 Nov 05 01:12:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5790(8162745) in scavenge on r816u03n04 Nov 05 01:12:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5791(8162746) in scavenge on r816u35n03 Nov 05 01:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5634(8161754) WEXITSTATUS 1 Nov 05 01:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5634(8161754) done Nov 05 01:13:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5561(8161215) WEXITSTATUS 1 Nov 05 01:13:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5561(8161215) done Nov 05 01:13:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5244(8158651) WEXITSTATUS 1 Nov 05 01:13:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5244(8158651) done Nov 05 01:13:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_27(8158659) WEXITSTATUS 0 Nov 05 01:13:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_27(8158659) done Nov 05 01:13:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_3(8160026) WEXITSTATUS 0 Nov 05 01:13:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_3(8160026) done Nov 05 01:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5295(8158743) WEXITSTATUS 1 Nov 05 01:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5295(8158743) done Nov 05 01:13:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3647(8162724) WEXITSTATUS 0 Nov 05 01:13:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3647(8162724) done Nov 05 01:13:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5189(8158552) WEXITSTATUS 1 Nov 05 01:13:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5189(8158552) done Nov 05 01:13:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5734(8162567) WEXITSTATUS 1 Nov 05 01:13:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5734(8162567) done Nov 05 01:13:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1267(8135205) WEXITSTATUS 0 Nov 05 01:13:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1267(8135205) done Nov 05 01:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157656 WEXITSTATUS 0 Nov 05 01:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157656 done Nov 05 01:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_12(8160035) WEXITSTATUS 0 Nov 05 01:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_12(8160035) done Nov 05 01:13:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_41(8160064) WEXITSTATUS 0 Nov 05 01:13:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_41(8160064) done Nov 05 01:13:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_58(8160081) WEXITSTATUS 0 Nov 05 01:13:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_58(8160081) done Nov 05 01:13:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_56(8160079) WEXITSTATUS 0 Nov 05 01:13:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_56(8160079) done Nov 05 01:13:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_5(8160028) WEXITSTATUS 0 Nov 05 01:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_5(8160028) done Nov 05 01:13:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5552(8161042) WEXITSTATUS 1 Nov 05 01:13:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5552(8161042) done Nov 05 01:13:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5769(8162705) WEXITSTATUS 1 Nov 05 01:13:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5769(8162705) done Nov 05 01:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_2(8160025) WEXITSTATUS 0 Nov 05 01:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_2(8160025) done Nov 05 01:14:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159914 WEXITSTATUS 0 Nov 05 01:14:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159914 done Nov 05 01:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1258(8135163) WEXITSTATUS 0 Nov 05 01:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1258(8135163) done Nov 05 01:14:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_283(8160491) WEXITSTATUS 0 Nov 05 01:14:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_283(8160491) done Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:14:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:14:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_149(8160275) WEXITSTATUS 0 Nov 05 01:14:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_149(8160275) done Nov 05 01:14:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5585(8161405) WEXITSTATUS 1 Nov 05 01:14:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5585(8161405) done Nov 05 01:14:20 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:25 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:26 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:14:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_32(8160055) WEXITSTATUS 0 Nov 05 01:14:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_32(8160055) done Nov 05 01:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3646(8162712) WEXITSTATUS 0 Nov 05 01:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3646(8162712) done Nov 05 01:14:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_0(8160023) WEXITSTATUS 0 Nov 05 01:14:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_0(8160023) done Nov 05 01:14:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162747 InitPrio=21238 usec=3498 Nov 05 01:14:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_285(8160493) WEXITSTATUS 0 Nov 05 01:14:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_285(8160493) done Nov 05 01:14:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_96(8160119) WEXITSTATUS 0 Nov 05 01:14:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_96(8160119) done Nov 05 01:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_276(8160484) WEXITSTATUS 0 Nov 05 01:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_276(8160484) done Nov 05 01:15:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_10(8160033) WEXITSTATUS 0 Nov 05 01:15:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_10(8160033) done Nov 05 01:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_54(8160077) WEXITSTATUS 0 Nov 05 01:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_54(8160077) done Nov 05 01:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_50(8160073) WEXITSTATUS 0 Nov 05 01:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_50(8160073) done Nov 05 01:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_53(8160076) WEXITSTATUS 0 Nov 05 01:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_53(8160076) done Nov 05 01:15:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1502(8142737) WEXITSTATUS 0 Nov 05 01:15:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1502(8142737) done Nov 05 01:15:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_94(8160117) WEXITSTATUS 0 Nov 05 01:15:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_94(8160117) done Nov 05 01:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_134(8160260) WEXITSTATUS 0 Nov 05 01:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_134(8160260) done Nov 05 01:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_257(8160465) WEXITSTATUS 0 Nov 05 01:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_257(8160465) done Nov 05 01:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_299(8160507) WEXITSTATUS 0 Nov 05 01:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_299(8160507) done Nov 05 01:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_20(8160043) WEXITSTATUS 0 Nov 05 01:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_20(8160043) done Nov 05 01:15:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_295(8160503) WEXITSTATUS 0 Nov 05 01:15:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_295(8160503) done Nov 05 01:15:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5531(8160866) WEXITSTATUS 1 Nov 05 01:15:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5531(8160866) done Nov 05 01:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5569(8161223) WEXITSTATUS 1 Nov 05 01:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5569(8161223) done Nov 05 01:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5780(8162730) WEXITSTATUS 1 Nov 05 01:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5780(8162730) done Nov 05 01:15:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_280(8160488) WEXITSTATUS 0 Nov 05 01:15:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_280(8160488) done Nov 05 01:15:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162655 in day on r815u03n08 Nov 05 01:15:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3649(8162748) in gpu on r808u14n01 Nov 05 01:15:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3650(8162749) in gpu on r4513u23n01 Nov 05 01:15:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5486(8160135) WEXITSTATUS 1 Nov 05 01:15:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5486(8160135) done Nov 05 01:15:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1066(7573235) WEXITSTATUS 0 Nov 05 01:15:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1066(7573235) done Nov 05 01:15:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_623(8114892) WEXITSTATUS 0 Nov 05 01:15:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_623(8114892) done Nov 05 01:15:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1887(8162750) in day on r816u15n04 Nov 05 01:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_297(8160505) WEXITSTATUS 0 Nov 05 01:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_297(8160505) done Nov 05 01:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_133(8160259) WEXITSTATUS 0 Nov 05 01:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_133(8160259) done Nov 05 01:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_84(8160107) WEXITSTATUS 0 Nov 05 01:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_84(8160107) done Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5792(8162751) in scavenge on r906u01n02 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5793(8162752) in scavenge on r907u05n04 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5794(8162753) in scavenge on r917u01n02 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5795(8162754) in scavenge on r816u15n11 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5796(8162755) in scavenge on r805u06n04 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5797(8162756) in scavenge on r913u24n04 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5798(8162757) in scavenge on r815u29n02 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5799(8162758) in scavenge on r815u29n04 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5800(8162759) in scavenge on r816u29n02 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5801(8162760) in scavenge on r816u29n05 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5802(8162761) in scavenge on r816u03n01 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5803(8162762) in scavenge on r816u09n03 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5804(8162763) in scavenge on r816u15n03 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_11(8160034) WEXITSTATUS 0 Nov 05 01:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_11(8160034) done Nov 05 01:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3650(8162749) WEXITSTATUS 0 Nov 05 01:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3650(8162749) done Nov 05 01:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_262(8160470) WEXITSTATUS 0 Nov 05 01:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_262(8160470) done Nov 05 01:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_25(8160048) WEXITSTATUS 0 Nov 05 01:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_25(8160048) done Nov 05 01:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_112(8160238) WEXITSTATUS 0 Nov 05 01:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_112(8160238) done Nov 05 01:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_281(8160489) WEXITSTATUS 0 Nov 05 01:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_281(8160489) done Nov 05 01:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_13(8160036) WEXITSTATUS 0 Nov 05 01:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_13(8160036) done Nov 05 01:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_1(8160024) WEXITSTATUS 0 Nov 05 01:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_1(8160024) done Nov 05 01:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_135(8160261) WEXITSTATUS 0 Nov 05 01:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_135(8160261) done Nov 05 01:15:56 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8155688 Nov 05 01:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_298(8160506) WEXITSTATUS 0 Nov 05 01:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_298(8160506) done Nov 05 01:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_110(8160236) WEXITSTATUS 0 Nov 05 01:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_110(8160236) done Nov 05 01:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_136(8160262) WEXITSTATUS 0 Nov 05 01:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_136(8160262) done Nov 05 01:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_261(8160469) WEXITSTATUS 0 Nov 05 01:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_261(8160469) done Nov 05 01:16:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_137(8160263) WEXITSTATUS 0 Nov 05 01:16:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_137(8160263) done Nov 05 01:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_9(8160032) WEXITSTATUS 0 Nov 05 01:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_9(8160032) done Nov 05 01:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_39(8160062) WEXITSTATUS 0 Nov 05 01:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_39(8160062) done Nov 05 01:16:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_97(8160120) WEXITSTATUS 0 Nov 05 01:16:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_97(8160120) done Nov 05 01:16:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_7(8160030) WEXITSTATUS 0 Nov 05 01:16:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_7(8160030) done Nov 05 01:16:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_138(8160264) WEXITSTATUS 0 Nov 05 01:16:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_138(8160264) done Nov 05 01:16:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_38(8160061) WEXITSTATUS 0 Nov 05 01:16:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_38(8160061) done Nov 05 01:16:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_91(8160114) WEXITSTATUS 0 Nov 05 01:16:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_91(8160114) done Nov 05 01:16:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_296(8160504) WEXITSTATUS 0 Nov 05 01:16:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_296(8160504) done Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:16:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_8(8160031) WEXITSTATUS 0 Nov 05 01:16:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_8(8160031) done Nov 05 01:16:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_44(8160067) WEXITSTATUS 0 Nov 05 01:16:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_44(8160067) done Nov 05 01:16:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_263(8160471) WEXITSTATUS 0 Nov 05 01:16:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_263(8160471) done Nov 05 01:16:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_19(8160042) WEXITSTATUS 0 Nov 05 01:16:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_19(8160042) done Nov 05 01:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_43(8160066) WEXITSTATUS 0 Nov 05 01:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_43(8160066) done Nov 05 01:16:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_4(8160027) WEXITSTATUS 0 Nov 05 01:16:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_4(8160027) done Nov 05 01:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157545 WEXITSTATUS 0 Nov 05 01:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157545 done Nov 05 01:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_26(8160049) WEXITSTATUS 0 Nov 05 01:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_26(8160049) done Nov 05 01:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_256(8160464) WEXITSTATUS 0 Nov 05 01:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_256(8160464) done Nov 05 01:16:27 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1521(8123784) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:16:27 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2106(8125917) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:16:27 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3064(8129875) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:16:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_8(8156352) Nov 05 01:16:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_22(8160045) WEXITSTATUS 0 Nov 05 01:16:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_22(8160045) done Nov 05 01:16:29 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3064(8129875) Nov 05 01:16:29 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1521(8123784) Nov 05 01:16:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_184(8160310) WEXITSTATUS 0 Nov 05 01:16:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_184(8160310) done Nov 05 01:16:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2106(8125917) Nov 05 01:16:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_294(8160502) WEXITSTATUS 0 Nov 05 01:16:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_294(8160502) done Nov 05 01:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_92(8160115) WEXITSTATUS 0 Nov 05 01:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_92(8160115) done Nov 05 01:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133208 WEXITSTATUS 0 Nov 05 01:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133208 done Nov 05 01:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_284(8160492) WEXITSTATUS 0 Nov 05 01:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_284(8160492) done Nov 05 01:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_279(8160487) WEXITSTATUS 0 Nov 05 01:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_279(8160487) done Nov 05 01:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_48(8160071) WEXITSTATUS 0 Nov 05 01:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_48(8160071) done Nov 05 01:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_34(8160057) WEXITSTATUS 0 Nov 05 01:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_34(8160057) done Nov 05 01:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_288(8160496) WEXITSTATUS 0 Nov 05 01:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_288(8160496) done Nov 05 01:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_287(8160495) WEXITSTATUS 0 Nov 05 01:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_287(8160495) done Nov 05 01:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_89(8160112) WEXITSTATUS 0 Nov 05 01:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_89(8160112) done Nov 05 01:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_120(8160246) WEXITSTATUS 0 Nov 05 01:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_120(8160246) done Nov 05 01:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_99(8160122) WEXITSTATUS 0 Nov 05 01:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_99(8160122) done Nov 05 01:16:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_42(8160065) WEXITSTATUS 0 Nov 05 01:16:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_42(8160065) done Nov 05 01:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_55(8160078) WEXITSTATUS 0 Nov 05 01:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_55(8160078) done Nov 05 01:16:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_103(8160229) WEXITSTATUS 0 Nov 05 01:16:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_103(8160229) done Nov 05 01:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_46(8160069) WEXITSTATUS 0 Nov 05 01:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_46(8160069) done Nov 05 01:16:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_267(8160475) WEXITSTATUS 0 Nov 05 01:16:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_267(8160475) done Nov 05 01:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_71(8160094) WEXITSTATUS 0 Nov 05 01:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_71(8160094) done Nov 05 01:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_52(8160075) WEXITSTATUS 0 Nov 05 01:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_52(8160075) done Nov 05 01:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_95(8160118) WEXITSTATUS 0 Nov 05 01:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_95(8160118) done Nov 05 01:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5296(8158744) WEXITSTATUS 1 Nov 05 01:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5296(8158744) done Nov 05 01:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_101(8160227) WEXITSTATUS 0 Nov 05 01:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_101(8160227) done Nov 05 01:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_274(8160482) WEXITSTATUS 0 Nov 05 01:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_274(8160482) done Nov 05 01:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_275(8160483) WEXITSTATUS 0 Nov 05 01:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_275(8160483) done Nov 05 01:16:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_47(8160070) WEXITSTATUS 0 Nov 05 01:16:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_47(8160070) done Nov 05 01:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_113(8160239) WEXITSTATUS 0 Nov 05 01:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_113(8160239) done Nov 05 01:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_146(8160272) WEXITSTATUS 0 Nov 05 01:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_146(8160272) done Nov 05 01:16:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_185(8160311) WEXITSTATUS 0 Nov 05 01:16:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_185(8160311) done Nov 05 01:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_35(8160058) WEXITSTATUS 0 Nov 05 01:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_35(8160058) done Nov 05 01:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_272(8160480) WEXITSTATUS 0 Nov 05 01:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_272(8160480) done Nov 05 01:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_86(8160109) WEXITSTATUS 0 Nov 05 01:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_86(8160109) done Nov 05 01:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_141(8160267) WEXITSTATUS 0 Nov 05 01:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_141(8160267) done Nov 05 01:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_192(8160318) WEXITSTATUS 0 Nov 05 01:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_192(8160318) done Nov 05 01:17:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_18(8160041) WEXITSTATUS 0 Nov 05 01:17:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_18(8160041) done Nov 05 01:17:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:17:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:17:08 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:17:09 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:17:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:17:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_45(8160068) WEXITSTATUS 0 Nov 05 01:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_45(8160068) done Nov 05 01:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_282(8160490) WEXITSTATUS 0 Nov 05 01:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_282(8160490) done Nov 05 01:17:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_270(8160478) WEXITSTATUS 0 Nov 05 01:17:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_270(8160478) done Nov 05 01:17:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_29(8160052) WEXITSTATUS 0 Nov 05 01:17:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_29(8160052) done Nov 05 01:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_16(8160039) WEXITSTATUS 0 Nov 05 01:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_16(8160039) done Nov 05 01:17:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_87(8160110) WEXITSTATUS 0 Nov 05 01:17:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_87(8160110) done Nov 05 01:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_286(8160494) WEXITSTATUS 0 Nov 05 01:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_286(8160494) done Nov 05 01:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_111(8160237) WEXITSTATUS 0 Nov 05 01:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_111(8160237) done Nov 05 01:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_197(8160323) WEXITSTATUS 0 Nov 05 01:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_197(8160323) done Nov 05 01:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_169(8160295) WEXITSTATUS 0 Nov 05 01:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_169(8160295) done Nov 05 01:17:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5574(8161228) WEXITSTATUS 1 Nov 05 01:17:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5574(8161228) done Nov 05 01:17:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_118(8160244) WEXITSTATUS 0 Nov 05 01:17:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_118(8160244) done Nov 05 01:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5559(8161049) WEXITSTATUS 1 Nov 05 01:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5559(8161049) done Nov 05 01:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_6(8160029) WEXITSTATUS 0 Nov 05 01:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_6(8160029) done Nov 05 01:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_85(8160108) WEXITSTATUS 0 Nov 05 01:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_85(8160108) done Nov 05 01:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_107(8160233) WEXITSTATUS 0 Nov 05 01:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_107(8160233) done Nov 05 01:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_318(8160611) WEXITSTATUS 0 Nov 05 01:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_318(8160611) done Nov 05 01:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_306(8160599) WEXITSTATUS 0 Nov 05 01:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_306(8160599) done Nov 05 01:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_143(8160269) WEXITSTATUS 0 Nov 05 01:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_143(8160269) done Nov 05 01:17:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_40(8160063) WEXITSTATUS 0 Nov 05 01:17:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_40(8160063) done Nov 05 01:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_90(8160113) WEXITSTATUS 0 Nov 05 01:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_90(8160113) done Nov 05 01:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_259(8160467) WEXITSTATUS 0 Nov 05 01:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_259(8160467) done Nov 05 01:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5599(8161686) WEXITSTATUS 1 Nov 05 01:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5599(8161686) done Nov 05 01:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_277(8160485) WEXITSTATUS 0 Nov 05 01:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_277(8160485) done Nov 05 01:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5301(8158763) WEXITSTATUS 1 Nov 05 01:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5301(8158763) done Nov 05 01:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_140(8160266) WEXITSTATUS 0 Nov 05 01:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_140(8160266) done Nov 05 01:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_139(8160265) WEXITSTATUS 0 Nov 05 01:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_139(8160265) done Nov 05 01:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_316(8160609) WEXITSTATUS 0 Nov 05 01:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_316(8160609) done Nov 05 01:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5646(8161939) WEXITSTATUS 1 Nov 05 01:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5646(8161939) done Nov 05 01:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_108(8160234) WEXITSTATUS 0 Nov 05 01:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_108(8160234) done Nov 05 01:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_77(8160100) WEXITSTATUS 0 Nov 05 01:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_77(8160100) done Nov 05 01:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5072(8158384) WEXITSTATUS 1 Nov 05 01:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5072(8158384) done Nov 05 01:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_174(8160300) WEXITSTATUS 0 Nov 05 01:17:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_174(8160300) done Nov 05 01:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_170(8160296) WEXITSTATUS 0 Nov 05 01:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_170(8160296) done Nov 05 01:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_126(8160252) WEXITSTATUS 0 Nov 05 01:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_126(8160252) done Nov 05 01:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_179(8160305) WEXITSTATUS 0 Nov 05 01:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_179(8160305) done Nov 05 01:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_190(8160316) WEXITSTATUS 0 Nov 05 01:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_190(8160316) done Nov 05 01:17:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_21(8160044) WEXITSTATUS 0 Nov 05 01:17:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_21(8160044) done Nov 05 01:17:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_125(8160251) WEXITSTATUS 0 Nov 05 01:17:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_125(8160251) done Nov 05 01:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_258(8160466) WEXITSTATUS 0 Nov 05 01:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_258(8160466) done Nov 05 01:17:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_273(8160481) WEXITSTATUS 0 Nov 05 01:17:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_273(8160481) done Nov 05 01:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_28(8160051) WEXITSTATUS 0 Nov 05 01:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_28(8160051) done Nov 05 01:17:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_155(8160281) WEXITSTATUS 0 Nov 05 01:17:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_155(8160281) done Nov 05 01:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_319(8160612) WEXITSTATUS 0 Nov 05 01:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_319(8160612) done Nov 05 01:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_24(8160047) WEXITSTATUS 0 Nov 05 01:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_24(8160047) done Nov 05 01:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_117(8160243) WEXITSTATUS 0 Nov 05 01:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_117(8160243) done Nov 05 01:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_88(8160111) WEXITSTATUS 0 Nov 05 01:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_88(8160111) done Nov 05 01:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157669 WEXITSTATUS 0 Nov 05 01:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157669 done Nov 05 01:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_98(8160121) WEXITSTATUS 0 Nov 05 01:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_98(8160121) done Nov 05 01:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_23(8160046) WEXITSTATUS 0 Nov 05 01:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_23(8160046) done Nov 05 01:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_122(8160248) WEXITSTATUS 0 Nov 05 01:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_122(8160248) done Nov 05 01:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_36(8160059) WEXITSTATUS 0 Nov 05 01:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_36(8160059) done Nov 05 01:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_31(8160054) WEXITSTATUS 0 Nov 05 01:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_31(8160054) done Nov 05 01:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_313(8160606) WEXITSTATUS 0 Nov 05 01:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_313(8160606) done Nov 05 01:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_182(8160308) WEXITSTATUS 0 Nov 05 01:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_182(8160308) done Nov 05 01:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_186(8160312) WEXITSTATUS 0 Nov 05 01:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_186(8160312) done Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_191(8160317) WEXITSTATUS 0 Nov 05 01:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_191(8160317) done Nov 05 01:18:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_167(8160293) WEXITSTATUS 0 Nov 05 01:18:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_167(8160293) done Nov 05 01:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_30(8160053) WEXITSTATUS 0 Nov 05 01:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_30(8160053) done Nov 05 01:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_159(8160285) WEXITSTATUS 0 Nov 05 01:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_159(8160285) done Nov 05 01:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_78(8160101) WEXITSTATUS 0 Nov 05 01:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_78(8160101) done Nov 05 01:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_116(8160242) WEXITSTATUS 0 Nov 05 01:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_116(8160242) done Nov 05 01:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_303(8160596) WEXITSTATUS 0 Nov 05 01:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_303(8160596) done Nov 05 01:18:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162648 in day on r807u03n01 Nov 05 01:18:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162747 in day on r918u09n02,r918u36n02 Nov 05 01:18:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3651(8162764) in gpu on r4513u23n01 Nov 05 01:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_102(8160228) WEXITSTATUS 0 Nov 05 01:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_102(8160228) done Nov 05 01:18:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3623(8161683) WEXITSTATUS 0 Nov 05 01:18:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3623(8161683) done Nov 05 01:18:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 01:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_119(8160245) WEXITSTATUS 0 Nov 05 01:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_119(8160245) done Nov 05 01:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_309(8160602) WEXITSTATUS 0 Nov 05 01:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_309(8160602) done Nov 05 01:18:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_9(8156474) Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5805(8162765) in scavenge on r802u02n03 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5806(8162766) in scavenge on r802u11n03 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5807(8162767) in scavenge on r815u09n12 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5808(8162768) in scavenge on r816u15n09 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5809(8162769) in scavenge on r815u29n11 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5810(8162770) in scavenge on r815u29n12 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5811(8162771) in scavenge on r807u11n04 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5812(8162772) in scavenge on r904u07n03 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5813(8162773) in scavenge on r904u07n04 Nov 05 01:18:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5814(8162774) in scavenge on r805u34n01 Nov 05 01:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_80(8160103) WEXITSTATUS 0 Nov 05 01:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_80(8160103) done Nov 05 01:18:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_132(8160258) WEXITSTATUS 0 Nov 05 01:18:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_132(8160258) done Nov 05 01:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154643 WEXITSTATUS 0 Nov 05 01:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154643 done Nov 05 01:18:37 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 01:18:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4796(8157692) WEXITSTATUS 1 Nov 05 01:18:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4796(8157692) done Nov 05 01:18:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_154(8160280) WEXITSTATUS 0 Nov 05 01:18:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_154(8160280) done Nov 05 01:18:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_93(8160116) WEXITSTATUS 0 Nov 05 01:18:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_93(8160116) done Nov 05 01:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_172(8160298) WEXITSTATUS 0 Nov 05 01:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_172(8160298) done Nov 05 01:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_187(8160313) WEXITSTATUS 0 Nov 05 01:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_187(8160313) done Nov 05 01:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_196(8160322) WEXITSTATUS 0 Nov 05 01:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_196(8160322) done Nov 05 01:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5507(8160510) WEXITSTATUS 1 Nov 05 01:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5507(8160510) done Nov 05 01:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_292(8160500) WEXITSTATUS 0 Nov 05 01:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_292(8160500) done Nov 05 01:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_37(8160060) WEXITSTATUS 0 Nov 05 01:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_37(8160060) done Nov 05 01:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5776(8162718) WEXITSTATUS 1 Nov 05 01:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5776(8162718) done Nov 05 01:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_82(8160105) WEXITSTATUS 0 Nov 05 01:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_82(8160105) done Nov 05 01:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_127(8160253) WEXITSTATUS 0 Nov 05 01:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_127(8160253) done Nov 05 01:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_278(8160486) WEXITSTATUS 0 Nov 05 01:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_278(8160486) done Nov 05 01:18:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159913 WEXITSTATUS 0 Nov 05 01:18:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159913 done Nov 05 01:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_330(8160623) WEXITSTATUS 0 Nov 05 01:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_330(8160623) done Nov 05 01:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_326(8160619) WEXITSTATUS 0 Nov 05 01:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_326(8160619) done Nov 05 01:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_74(8160097) WEXITSTATUS 0 Nov 05 01:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_74(8160097) done Nov 05 01:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_266(8160474) WEXITSTATUS 0 Nov 05 01:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_266(8160474) done Nov 05 01:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_320(8160613) WEXITSTATUS 0 Nov 05 01:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_320(8160613) done Nov 05 01:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_130(8160256) WEXITSTATUS 0 Nov 05 01:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_130(8160256) done Nov 05 01:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_268(8160476) WEXITSTATUS 0 Nov 05 01:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_268(8160476) done Nov 05 01:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_168(8160294) WEXITSTATUS 0 Nov 05 01:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_168(8160294) done Nov 05 01:18:57 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2138(8125954) Nov 05 01:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_328(8160621) WEXITSTATUS 0 Nov 05 01:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_328(8160621) done Nov 05 01:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_150(8160276) WEXITSTATUS 0 Nov 05 01:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_150(8160276) done Nov 05 01:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_144(8160270) WEXITSTATUS 0 Nov 05 01:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_144(8160270) done Nov 05 01:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_315(8160608) WEXITSTATUS 0 Nov 05 01:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_315(8160608) done Nov 05 01:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_73(8160096) WEXITSTATUS 0 Nov 05 01:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_73(8160096) done Nov 05 01:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_325(8160618) WEXITSTATUS 0 Nov 05 01:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_325(8160618) done Nov 05 01:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_329(8160622) WEXITSTATUS 0 Nov 05 01:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_329(8160622) done Nov 05 01:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_177(8160303) WEXITSTATUS 0 Nov 05 01:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_177(8160303) done Nov 05 01:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_786(8161381) WEXITSTATUS 0 Nov 05 01:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_786(8161381) done Nov 05 01:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_199(8160325) WEXITSTATUS 0 Nov 05 01:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_199(8160325) done Nov 05 01:19:04 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:19:05 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:19:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:19:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_784(8161379) WEXITSTATUS 0 Nov 05 01:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_784(8161379) done Nov 05 01:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_76(8160099) WEXITSTATUS 0 Nov 05 01:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_76(8160099) done Nov 05 01:19:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_145(8160271) WEXITSTATUS 0 Nov 05 01:19:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_145(8160271) done Nov 05 01:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_250(8160458) WEXITSTATUS 0 Nov 05 01:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_250(8160458) done Nov 05 01:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_790(8161385) WEXITSTATUS 0 Nov 05 01:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_790(8161385) done Nov 05 01:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_83(8160106) WEXITSTATUS 0 Nov 05 01:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_83(8160106) done Nov 05 01:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131872 WEXITSTATUS 0 Nov 05 01:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131872 done Nov 05 01:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_787(8161382) WEXITSTATUS 0 Nov 05 01:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_787(8161382) done Nov 05 01:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_269(8160477) WEXITSTATUS 0 Nov 05 01:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_269(8160477) done Nov 05 01:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_128(8160254) WEXITSTATUS 0 Nov 05 01:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_128(8160254) done Nov 05 01:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157666 WEXITSTATUS 0 Nov 05 01:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157666 done Nov 05 01:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133211 WEXITSTATUS 0 Nov 05 01:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133211 done Nov 05 01:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_49(8160072) WEXITSTATUS 0 Nov 05 01:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_49(8160072) done Nov 05 01:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_792(8161387) WEXITSTATUS 0 Nov 05 01:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_792(8161387) done Nov 05 01:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_230(8160438) WEXITSTATUS 0 Nov 05 01:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_230(8160438) done Nov 05 01:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_166(8160292) WEXITSTATUS 0 Nov 05 01:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_166(8160292) done Nov 05 01:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_81(8160104) WEXITSTATUS 0 Nov 05 01:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_81(8160104) done Nov 05 01:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_79(8160102) WEXITSTATUS 0 Nov 05 01:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_79(8160102) done Nov 05 01:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157668 WEXITSTATUS 0 Nov 05 01:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157668 done Nov 05 01:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_310(8160603) WEXITSTATUS 0 Nov 05 01:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_310(8160603) done Nov 05 01:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_798(8161393) WEXITSTATUS 0 Nov 05 01:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_798(8161393) done Nov 05 01:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_151(8160277) WEXITSTATUS 0 Nov 05 01:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_151(8160277) done Nov 05 01:19:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_10(8156476) Nov 05 01:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_158(8160284) WEXITSTATUS 0 Nov 05 01:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_158(8160284) done Nov 05 01:19:28 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5329(8158902) WEXITSTATUS 1 Nov 05 01:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5329(8158902) done Nov 05 01:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3651(8162764) WEXITSTATUS 0 Nov 05 01:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3651(8162764) done Nov 05 01:19:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5810(8162770) WEXITSTATUS 1 Nov 05 01:19:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5810(8162770) done Nov 05 01:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_180(8160306) WEXITSTATUS 0 Nov 05 01:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_180(8160306) done Nov 05 01:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_254(8160462) WEXITSTATUS 0 Nov 05 01:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_254(8160462) done Nov 05 01:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_289(8160497) WEXITSTATUS 0 Nov 05 01:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_289(8160497) done Nov 05 01:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_327(8160620) WEXITSTATUS 0 Nov 05 01:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_327(8160620) done Nov 05 01:19:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_324(8160617) WEXITSTATUS 0 Nov 05 01:19:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_324(8160617) done Nov 05 01:19:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_332(8160625) WEXITSTATUS 0 Nov 05 01:19:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_332(8160625) done Nov 05 01:19:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_33(8160056) WEXITSTATUS 0 Nov 05 01:19:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_33(8160056) done Nov 05 01:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_75(8160098) WEXITSTATUS 0 Nov 05 01:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_75(8160098) done Nov 05 01:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_331(8160624) WEXITSTATUS 0 Nov 05 01:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_331(8160624) done Nov 05 01:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162694 WEXITSTATUS 0 Nov 05 01:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162694 done Nov 05 01:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5314(8158842) WEXITSTATUS 1 Nov 05 01:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5314(8158842) done Nov 05 01:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_14(8160037) WEXITSTATUS 0 Nov 05 01:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_14(8160037) done Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162695 NodeList=r801u04n01 #CPUs=12 Partition=pi_esi Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_152(8160278) WEXITSTATUS 0 Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_152(8160278) done Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_160(8160286) WEXITSTATUS 0 Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_160(8160286) done Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_793(8161388) WEXITSTATUS 0 Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_793(8161388) done Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_131(8160257) WEXITSTATUS 0 Nov 05 01:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_131(8160257) done Nov 05 01:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_264(8160472) WEXITSTATUS 0 Nov 05 01:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_264(8160472) done Nov 05 01:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_260(8160468) WEXITSTATUS 0 Nov 05 01:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_260(8160468) done Nov 05 01:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_124(8160250) WEXITSTATUS 0 Nov 05 01:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_124(8160250) done Nov 05 01:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_311(8160604) WEXITSTATUS 0 Nov 05 01:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_311(8160604) done Nov 05 01:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_290(8160498) WEXITSTATUS 0 Nov 05 01:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_290(8160498) done Nov 05 01:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_121(8160247) WEXITSTATUS 0 Nov 05 01:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_121(8160247) done Nov 05 01:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_322(8160615) WEXITSTATUS 0 Nov 05 01:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_322(8160615) done Nov 05 01:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_198(8160324) WEXITSTATUS 0 Nov 05 01:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_198(8160324) done Nov 05 01:19:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_72(8160095) WEXITSTATUS 0 Nov 05 01:19:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_72(8160095) done Nov 05 01:19:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_785(8161380) WEXITSTATUS 0 Nov 05 01:19:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_785(8161380) done Nov 05 01:19:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_163(8160289) WEXITSTATUS 0 Nov 05 01:19:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_163(8160289) done Nov 05 01:19:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_0(8158573) WEXITSTATUS 0 Nov 05 01:19:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_0(8158573) done Nov 05 01:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_171(8160297) WEXITSTATUS 0 Nov 05 01:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_171(8160297) done Nov 05 01:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_783(8161378) WEXITSTATUS 0 Nov 05 01:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_783(8161378) done Nov 05 01:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_251(8160459) WEXITSTATUS 0 Nov 05 01:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_251(8160459) done Nov 05 01:20:00 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 01:20:01 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 01:20:01 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 01:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_323(8160616) WEXITSTATUS 0 Nov 05 01:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_323(8160616) done Nov 05 01:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_317(8160610) WEXITSTATUS 0 Nov 05 01:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_317(8160610) done Nov 05 01:20:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_795(8161390) WEXITSTATUS 0 Nov 05 01:20:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_795(8161390) done Nov 05 01:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_162(8160288) WEXITSTATUS 0 Nov 05 01:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_162(8160288) done Nov 05 01:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_291(8160499) WEXITSTATUS 0 Nov 05 01:20:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_291(8160499) done Nov 05 01:20:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133319 WEXITSTATUS 0 Nov 05 01:20:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133319 done Nov 05 01:20:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_308(8160601) WEXITSTATUS 0 Nov 05 01:20:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_308(8160601) done Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_797(8161392) WEXITSTATUS 0 Nov 05 01:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_797(8161392) done Nov 05 01:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_314(8160607) WEXITSTATUS 0 Nov 05 01:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_314(8160607) done Nov 05 01:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_794(8161389) WEXITSTATUS 0 Nov 05 01:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_794(8161389) done Nov 05 01:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_161(8160287) WEXITSTATUS 0 Nov 05 01:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_161(8160287) done Nov 05 01:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_788(8161383) WEXITSTATUS 0 Nov 05 01:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_788(8161383) done Nov 05 01:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_27(8160050) WEXITSTATUS 0 Nov 05 01:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_27(8160050) done Nov 05 01:20:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5783(8162733) WEXITSTATUS 1 Nov 05 01:20:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5783(8162733) done Nov 05 01:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_789(8161384) WEXITSTATUS 0 Nov 05 01:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_789(8161384) done Nov 05 01:20:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_157(8160283) WEXITSTATUS 0 Nov 05 01:20:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_157(8160283) done Nov 05 01:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_178(8160304) WEXITSTATUS 0 Nov 05 01:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_178(8160304) done Nov 05 01:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_175(8160301) WEXITSTATUS 0 Nov 05 01:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_175(8160301) done Nov 05 01:20:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_15(8160038) WEXITSTATUS 0 Nov 05 01:20:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_15(8160038) done Nov 05 01:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_302(8160595) WEXITSTATUS 0 Nov 05 01:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_302(8160595) done Nov 05 01:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_237(8160445) WEXITSTATUS 0 Nov 05 01:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_237(8160445) done Nov 05 01:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_799(8161394) WEXITSTATUS 0 Nov 05 01:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_799(8161394) done Nov 05 01:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_307(8160600) WEXITSTATUS 0 Nov 05 01:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_307(8160600) done Nov 05 01:20:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_238(8160446) WEXITSTATUS 0 Nov 05 01:20:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_238(8160446) done Nov 05 01:20:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_129(8160255) WEXITSTATUS 0 Nov 05 01:20:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_129(8160255) done Nov 05 01:20:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_796(8161391) WEXITSTATUS 0 Nov 05 01:20:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_796(8161391) done Nov 05 01:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4944(8158189) WEXITSTATUS 1 Nov 05 01:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4944(8158189) done Nov 05 01:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_100(8160226) WEXITSTATUS 0 Nov 05 01:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_100(8160226) done Nov 05 01:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_411(8160770) WEXITSTATUS 0 Nov 05 01:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_411(8160770) done Nov 05 01:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157667 WEXITSTATUS 0 Nov 05 01:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157667 done Nov 05 01:20:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_70(8160093) WEXITSTATUS 0 Nov 05 01:20:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_70(8160093) done Nov 05 01:20:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158374 WEXITSTATUS 0 Nov 05 01:20:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158374 done Nov 05 01:20:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_293(8160501) WEXITSTATUS 0 Nov 05 01:20:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_293(8160501) done Nov 05 01:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_304(8160597) WEXITSTATUS 0 Nov 05 01:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_304(8160597) done Nov 05 01:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5522(8160701) WEXITSTATUS 1 Nov 05 01:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5522(8160701) done Nov 05 01:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_156(8160282) WEXITSTATUS 0 Nov 05 01:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_156(8160282) done Nov 05 01:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_233(8160441) WEXITSTATUS 0 Nov 05 01:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_233(8160441) done Nov 05 01:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_777(8161372) WEXITSTATUS 0 Nov 05 01:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_777(8161372) done Nov 05 01:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_239(8160447) WEXITSTATUS 0 Nov 05 01:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_239(8160447) done Nov 05 01:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_195(8160321) WEXITSTATUS 0 Nov 05 01:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_195(8160321) done Nov 05 01:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_214(8160422) WEXITSTATUS 0 Nov 05 01:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_214(8160422) done Nov 05 01:21:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162657 in day on r815u03n05 Nov 05 01:21:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162658 in day on r815u35n01 Nov 05 01:21:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3652(8162775) in gpu on r808u11n01 Nov 05 01:21:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3653(8162776) in gpu on r4513u23n01 Nov 05 01:21:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_305(8160598) WEXITSTATUS 0 Nov 05 01:21:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_305(8160598) done Nov 05 01:21:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_252(8160460) WEXITSTATUS 0 Nov 05 01:21:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_252(8160460) done Nov 05 01:21:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133393 in gpu on r907u32n01 Nov 05 01:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5118(8158451) WEXITSTATUS 1 Nov 05 01:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5118(8158451) done Nov 05 01:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_232(8160440) WEXITSTATUS 0 Nov 05 01:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_232(8160440) done Nov 05 01:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3653(8162776) WEXITSTATUS 0 Nov 05 01:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3653(8162776) done Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5815(8162777) in scavenge on r801u04n01 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5816(8162778) in scavenge on r801u09n03 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5817(8162779) in scavenge on r802u06n03 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5818(8162780) in scavenge on r802u11n02 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5819(8162781) in scavenge on r815u15n11 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5820(8162782) in scavenge on r815u29n12 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5821(8162783) in scavenge on r816u03n10 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5822(8162784) in scavenge on r816u35n10 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5823(8162785) in scavenge on r907u32n01 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5824(8162786) in scavenge on r805u09n03 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5825(8162787) in scavenge on r807u03n01 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5826(8162788) in scavenge on r909u07n03 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5827(8162789) in scavenge on r815u23n04 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5828(8162790) in scavenge on r815u29n02 Nov 05 01:21:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5829(8162791) in scavenge on r815u29n04 Nov 05 01:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_217(8160425) WEXITSTATUS 0 Nov 05 01:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_217(8160425) done Nov 05 01:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_399(7572567) in scavenge on r816u03n10 Nov 05 01:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_242(8114508) in scavenge on r807u03n01 Nov 05 01:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_210(7572378) in scavenge on r815u29n02 Nov 05 01:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_211(7572379) in scavenge on r806u37n01 Nov 05 01:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_778(8161373) WEXITSTATUS 0 Nov 05 01:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_778(8161373) done Nov 05 01:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_247(8160455) WEXITSTATUS 0 Nov 05 01:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_247(8160455) done Nov 05 01:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_410(8160769) WEXITSTATUS 0 Nov 05 01:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_410(8160769) done Nov 05 01:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_255(8160463) WEXITSTATUS 0 Nov 05 01:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_255(8160463) done Nov 05 01:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_776(8161371) WEXITSTATUS 0 Nov 05 01:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_776(8161371) done Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_791(8161386) WEXITSTATUS 0 Nov 05 01:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_791(8161386) done Nov 05 01:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_173(8160299) WEXITSTATUS 0 Nov 05 01:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_173(8160299) done Nov 05 01:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_236(8160444) WEXITSTATUS 0 Nov 05 01:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_236(8160444) done Nov 05 01:21:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_781(8161376) WEXITSTATUS 0 Nov 05 01:21:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_781(8161376) done Nov 05 01:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_363(8160656) WEXITSTATUS 0 Nov 05 01:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_363(8160656) done Nov 05 01:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3652(8162775) WEXITSTATUS 0 Nov 05 01:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3652(8162775) done Nov 05 01:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_234(8160442) WEXITSTATUS 0 Nov 05 01:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_234(8160442) done Nov 05 01:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1450(8136670) WEXITSTATUS 0 Nov 05 01:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1450(8136670) done Nov 05 01:21:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_779(8161374) WEXITSTATUS 0 Nov 05 01:21:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_779(8161374) done Nov 05 01:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_181(8160307) WEXITSTATUS 0 Nov 05 01:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_181(8160307) done Nov 05 01:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_176(8160302) WEXITSTATUS 0 Nov 05 01:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_176(8160302) done Nov 05 01:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_188(8160314) WEXITSTATUS 0 Nov 05 01:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_188(8160314) done Nov 05 01:21:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162792 InitPrio=48651 usec=8369 Nov 05 01:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157700 WEXITSTATUS 0 Nov 05 01:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157700 done Nov 05 01:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_253(8160461) WEXITSTATUS 0 Nov 05 01:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_253(8160461) done Nov 05 01:21:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_218(8160426) WEXITSTATUS 0 Nov 05 01:21:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_218(8160426) done Nov 05 01:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_223(8160431) WEXITSTATUS 0 Nov 05 01:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_223(8160431) done Nov 05 01:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_212(8160420) WEXITSTATUS 0 Nov 05 01:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_212(8160420) done Nov 05 01:21:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162793 InitPrio=13461 usec=4841 Nov 05 01:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_164(8160290) WEXITSTATUS 0 Nov 05 01:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_164(8160290) done Nov 05 01:22:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_245(8160453) WEXITSTATUS 0 Nov 05 01:22:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_245(8160453) done Nov 05 01:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_401(8160760) WEXITSTATUS 0 Nov 05 01:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_401(8160760) done Nov 05 01:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_364(8160657) WEXITSTATUS 0 Nov 05 01:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_364(8160657) done Nov 05 01:22:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_402(8160761) WEXITSTATUS 0 Nov 05 01:22:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_402(8160761) done Nov 05 01:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_221(8160429) WEXITSTATUS 0 Nov 05 01:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_221(8160429) done Nov 05 01:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_248(8160456) WEXITSTATUS 0 Nov 05 01:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_248(8160456) done Nov 05 01:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_231(8160439) WEXITSTATUS 0 Nov 05 01:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_231(8160439) done Nov 05 01:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_216(8160424) WEXITSTATUS 0 Nov 05 01:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_216(8160424) done Nov 05 01:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_215(8160423) WEXITSTATUS 0 Nov 05 01:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_215(8160423) done Nov 05 01:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_775(8161370) WEXITSTATUS 0 Nov 05 01:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_775(8161370) done Nov 05 01:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_220(8160428) WEXITSTATUS 0 Nov 05 01:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_220(8160428) done Nov 05 01:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157704 WEXITSTATUS 0 Nov 05 01:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157704 done Nov 05 01:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_246(8160454) WEXITSTATUS 0 Nov 05 01:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_246(8160454) done Nov 05 01:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5383(8159321) WEXITSTATUS 1 Nov 05 01:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5383(8159321) done Nov 05 01:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1443(8136302) WEXITSTATUS 0 Nov 05 01:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1443(8136302) done Nov 05 01:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_142(8160268) WEXITSTATUS 0 Nov 05 01:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_142(8160268) done Nov 05 01:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_242(8160450) WEXITSTATUS 0 Nov 05 01:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_242(8160450) done Nov 05 01:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_376(8160669) WEXITSTATUS 0 Nov 05 01:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_376(8160669) done Nov 05 01:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_211(7572379) WEXITSTATUS 0 Nov 05 01:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_211(7572379) done Nov 05 01:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_994(7573163) WEXITSTATUS 0 Nov 05 01:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_994(7573163) done Nov 05 01:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_780(8161375) WEXITSTATUS 0 Nov 05 01:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_780(8161375) done Nov 05 01:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5627(8161747) WEXITSTATUS 1 Nov 05 01:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5627(8161747) done Nov 05 01:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_241(8160449) WEXITSTATUS 0 Nov 05 01:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_241(8160449) done Nov 05 01:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_228(8160436) WEXITSTATUS 0 Nov 05 01:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_228(8160436) done Nov 05 01:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_249(8160457) WEXITSTATUS 0 Nov 05 01:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_249(8160457) done Nov 05 01:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_387(8160680) WEXITSTATUS 0 Nov 05 01:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_387(8160680) done Nov 05 01:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5829(8162791) WEXITSTATUS 1 Nov 05 01:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5829(8162791) done Nov 05 01:22:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_333(8160626) WEXITSTATUS 0 Nov 05 01:22:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_333(8160626) done Nov 05 01:22:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_243(8160451) WEXITSTATUS 0 Nov 05 01:22:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_243(8160451) done Nov 05 01:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_321(8160614) WEXITSTATUS 0 Nov 05 01:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_321(8160614) done Nov 05 01:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_147(8160273) WEXITSTATUS 0 Nov 05 01:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_147(8160273) done Nov 05 01:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_229(8160437) WEXITSTATUS 0 Nov 05 01:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_229(8160437) done Nov 05 01:22:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5798(8162757) WEXITSTATUS 1 Nov 05 01:22:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5798(8162757) done Nov 05 01:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_219(8160427) WEXITSTATUS 0 Nov 05 01:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_219(8160427) done Nov 05 01:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3641(8162588) WEXITSTATUS 0 Nov 05 01:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3641(8162588) done Nov 05 01:22:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_403(8160762) WEXITSTATUS 0 Nov 05 01:22:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_403(8160762) done Nov 05 01:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_227(8160435) WEXITSTATUS 0 Nov 05 01:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_227(8160435) done Nov 05 01:23:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_367(8160660) WEXITSTATUS 0 Nov 05 01:23:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_367(8160660) done Nov 05 01:23:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_404(8160763) WEXITSTATUS 0 Nov 05 01:23:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_404(8160763) done Nov 05 01:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_240(8160448) WEXITSTATUS 0 Nov 05 01:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_240(8160448) done Nov 05 01:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_207(8160415) WEXITSTATUS 0 Nov 05 01:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_207(8160415) done Nov 05 01:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_405(8160764) WEXITSTATUS 0 Nov 05 01:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_405(8160764) done Nov 05 01:23:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4960(8158210) WEXITSTATUS 1 Nov 05 01:23:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4960(8158210) done Nov 05 01:23:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_224(8160432) WEXITSTATUS 0 Nov 05 01:23:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_224(8160432) done Nov 05 01:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_235(8160443) WEXITSTATUS 0 Nov 05 01:23:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_235(8160443) done Nov 05 01:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2108(8151784) WEXITSTATUS 0 Nov 05 01:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2108(8151784) done Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:23:26 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5579(8161399) WEXITSTATUS 1 Nov 05 01:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5579(8161399) done Nov 05 01:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5003(8158287) WEXITSTATUS 1 Nov 05 01:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5003(8158287) done Nov 05 01:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_385(8160678) WEXITSTATUS 0 Nov 05 01:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_385(8160678) done Nov 05 01:23:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_211(8160419) WEXITSTATUS 0 Nov 05 01:23:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_211(8160419) done Nov 05 01:23:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_343(8160636) WEXITSTATUS 0 Nov 05 01:23:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_343(8160636) done Nov 05 01:23:31 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162792 NodeList=r806u37n01 #CPUs=16 Partition=gpu Nov 05 01:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_226(8160434) WEXITSTATUS 0 Nov 05 01:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_226(8160434) done Nov 05 01:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157702 WEXITSTATUS 0 Nov 05 01:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157702 done Nov 05 01:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157701 WEXITSTATUS 0 Nov 05 01:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157701 done Nov 05 01:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_213(8160421) WEXITSTATUS 0 Nov 05 01:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_213(8160421) done Nov 05 01:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_377(8160670) WEXITSTATUS 0 Nov 05 01:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_377(8160670) done Nov 05 01:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_362(8160655) WEXITSTATUS 0 Nov 05 01:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_362(8160655) done Nov 05 01:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_379(8160672) WEXITSTATUS 0 Nov 05 01:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_379(8160672) done Nov 05 01:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_202(8160410) WEXITSTATUS 0 Nov 05 01:23:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_202(8160410) done Nov 05 01:23:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_334(8160627) WEXITSTATUS 0 Nov 05 01:23:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_334(8160627) done Nov 05 01:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_386(8160679) WEXITSTATUS 0 Nov 05 01:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_386(8160679) done Nov 05 01:23:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162793 in pi_balou on r911u09n01 Nov 05 01:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_344(8160637) WEXITSTATUS 0 Nov 05 01:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_344(8160637) done Nov 05 01:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_201(8160409) WEXITSTATUS 0 Nov 05 01:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_201(8160409) done Nov 05 01:23:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3654(8162794) in gpu on r808u08n01 Nov 05 01:23:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3655(8162795) in gpu on r808u11n01 Nov 05 01:23:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3656(8162796) in gpu on r4513u23n01 Nov 05 01:23:58 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1105(7573274) Nov 05 01:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_222(8160430) WEXITSTATUS 0 Nov 05 01:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_222(8160430) done Nov 05 01:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_774(8161369) WEXITSTATUS 0 Nov 05 01:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_774(8161369) done Nov 05 01:24:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 01:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_209(8160417) WEXITSTATUS 0 Nov 05 01:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_209(8160417) done Nov 05 01:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_104(8160230) WEXITSTATUS 0 Nov 05 01:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_104(8160230) done Nov 05 01:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133318 WEXITSTATUS 0 Nov 05 01:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133318 done Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5830(8162797) in scavenge on r801u27n02 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5831(8162798) in scavenge on r802u02n03 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5832(8162799) in scavenge on r903u07n01 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5833(8162800) in scavenge on r915u32n01 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5834(8162801) in scavenge on r4513u23n01 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5835(8162802) in scavenge on r904u10n01 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5836(8162803) in scavenge on r815u09n04 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5837(8162804) in scavenge on r816u35n01 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5838(8162805) in scavenge on r815u29n02 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5839(8162806) in scavenge on r815u29n04 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5840(8162807) in scavenge on r815u35n03 Nov 05 01:24:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5841(8162808) in scavenge on r815u35n04 Nov 05 01:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_342(8160635) WEXITSTATUS 0 Nov 05 01:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_342(8160635) done Nov 05 01:24:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_375(8160668) WEXITSTATUS 0 Nov 05 01:24:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_375(8160668) done Nov 05 01:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_148(8160274) WEXITSTATUS 0 Nov 05 01:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_148(8160274) done Nov 05 01:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_361(8160654) WEXITSTATUS 0 Nov 05 01:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_361(8160654) done Nov 05 01:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_336(8160629) WEXITSTATUS 0 Nov 05 01:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_336(8160629) done Nov 05 01:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_32(8158664) WEXITSTATUS 0 Nov 05 01:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_32(8158664) done Nov 05 01:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_432(8160791) WEXITSTATUS 0 Nov 05 01:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_432(8160791) done Nov 05 01:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_368(8160661) WEXITSTATUS 0 Nov 05 01:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_368(8160661) done Nov 05 01:24:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157707 WEXITSTATUS 0 Nov 05 01:24:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157707 done Nov 05 01:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154629 WEXITSTATUS 0 Nov 05 01:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154629 done Nov 05 01:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_983(8161652) WEXITSTATUS 0 Nov 05 01:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_983(8161652) done Nov 05 01:24:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3959(8146002) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:24:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4734(8156863) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:24:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3959(8146002) Nov 05 01:24:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4734(8156863) Nov 05 01:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157671 WEXITSTATUS 0 Nov 05 01:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157671 done Nov 05 01:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_340(8160633) WEXITSTATUS 0 Nov 05 01:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_340(8160633) done Nov 05 01:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_339(8160632) WEXITSTATUS 0 Nov 05 01:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_339(8160632) done Nov 05 01:24:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_503(8160938) WEXITSTATUS 0 Nov 05 01:24:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_503(8160938) done Nov 05 01:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162793 WEXITSTATUS 1 Nov 05 01:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162793 done Nov 05 01:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_225(8160433) WEXITSTATUS 0 Nov 05 01:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_225(8160433) done Nov 05 01:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_114(8160240) WEXITSTATUS 0 Nov 05 01:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_114(8160240) done Nov 05 01:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_301(8160594) WEXITSTATUS 0 Nov 05 01:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_301(8160594) done Nov 05 01:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_380(8160673) WEXITSTATUS 0 Nov 05 01:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_380(8160673) done Nov 05 01:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3656(8162796) WEXITSTATUS 0 Nov 05 01:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3656(8162796) done Nov 05 01:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_358(8160651) WEXITSTATUS 0 Nov 05 01:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_358(8160651) done Nov 05 01:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_203(8160411) WEXITSTATUS 0 Nov 05 01:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_203(8160411) done Nov 05 01:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_370(8160663) WEXITSTATUS 0 Nov 05 01:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_370(8160663) done Nov 05 01:24:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_389(8160682) WEXITSTATUS 0 Nov 05 01:24:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_389(8160682) done Nov 05 01:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_366(8160659) WEXITSTATUS 0 Nov 05 01:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_366(8160659) done Nov 05 01:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_388(8160681) WEXITSTATUS 0 Nov 05 01:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_388(8160681) done Nov 05 01:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_475(8160834) WEXITSTATUS 0 Nov 05 01:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_475(8160834) done Nov 05 01:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_193(8160319) WEXITSTATUS 0 Nov 05 01:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_193(8160319) done Nov 05 01:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_433(8160792) WEXITSTATUS 0 Nov 05 01:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_433(8160792) done Nov 05 01:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_381(8160674) WEXITSTATUS 0 Nov 05 01:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_381(8160674) done Nov 05 01:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3655(8162795) WEXITSTATUS 0 Nov 05 01:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3655(8162795) done Nov 05 01:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_493(8160852) WEXITSTATUS 0 Nov 05 01:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_493(8160852) done Nov 05 01:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_206(8160414) WEXITSTATUS 0 Nov 05 01:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_206(8160414) done Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:25:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:25:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:25:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_205(8160413) WEXITSTATUS 0 Nov 05 01:25:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_205(8160413) done Nov 05 01:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_378(8160671) WEXITSTATUS 0 Nov 05 01:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_378(8160671) done Nov 05 01:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_359(8160652) WEXITSTATUS 0 Nov 05 01:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_359(8160652) done Nov 05 01:25:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1332(8135433) WEXITSTATUS 0 Nov 05 01:25:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1332(8135433) done Nov 05 01:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_383(8160676) WEXITSTATUS 0 Nov 05 01:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_383(8160676) done Nov 05 01:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_408(8160767) WEXITSTATUS 0 Nov 05 01:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_408(8160767) done Nov 05 01:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_200(8160408) WEXITSTATUS 0 Nov 05 01:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_200(8160408) done Nov 05 01:25:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154639 WEXITSTATUS 0 Nov 05 01:25:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154639 done Nov 05 01:25:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5320(8158848) WEXITSTATUS 1 Nov 05 01:25:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5320(8158848) done Nov 05 01:25:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_382(8160675) WEXITSTATUS 0 Nov 05 01:25:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_382(8160675) done Nov 05 01:25:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162809 InitPrio=13461 usec=3725 Nov 05 01:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_988(8161657) WEXITSTATUS 0 Nov 05 01:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_988(8161657) done Nov 05 01:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_384(8160677) WEXITSTATUS 0 Nov 05 01:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_384(8160677) done Nov 05 01:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_341(8160634) WEXITSTATUS 0 Nov 05 01:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_341(8160634) done Nov 05 01:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_467(8160826) WEXITSTATUS 0 Nov 05 01:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_467(8160826) done Nov 05 01:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_782(8161377) WEXITSTATUS 0 Nov 05 01:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_782(8161377) done Nov 05 01:26:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5643(8161936) WEXITSTATUS 1 Nov 05 01:26:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5643(8161936) done Nov 05 01:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_982(8161651) WEXITSTATUS 0 Nov 05 01:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_982(8161651) done Nov 05 01:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_208(8160416) WEXITSTATUS 0 Nov 05 01:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_208(8160416) done Nov 05 01:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_460(8160819) WEXITSTATUS 0 Nov 05 01:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_460(8160819) done Nov 05 01:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_105(8160231) WEXITSTATUS 0 Nov 05 01:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_105(8160231) done Nov 05 01:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154603 WEXITSTATUS 0 Nov 05 01:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154603 done Nov 05 01:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_977(8161646) WEXITSTATUS 0 Nov 05 01:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_977(8161646) done Nov 05 01:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_365(8160658) WEXITSTATUS 0 Nov 05 01:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_365(8160658) done Nov 05 01:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_391(8160684) WEXITSTATUS 0 Nov 05 01:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_391(8160684) done Nov 05 01:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162736 WEXITSTATUS 0 Nov 05 01:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162736 done Nov 05 01:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_372(8160665) WEXITSTATUS 0 Nov 05 01:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_372(8160665) done Nov 05 01:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_390(8160683) WEXITSTATUS 0 Nov 05 01:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_390(8160683) done Nov 05 01:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5577(8161231) WEXITSTATUS 1 Nov 05 01:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5577(8161231) done Nov 05 01:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159916 WEXITSTATUS 0 Nov 05 01:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159916 done Nov 05 01:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_204(8160412) WEXITSTATUS 0 Nov 05 01:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_204(8160412) done Nov 05 01:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157705 WEXITSTATUS 0 Nov 05 01:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157705 done Nov 05 01:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_996(8161665) WEXITSTATUS 0 Nov 05 01:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_996(8161665) done Nov 05 01:26:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_995(8161664) WEXITSTATUS 0 Nov 05 01:26:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_995(8161664) done Nov 05 01:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133212 WEXITSTATUS 0 Nov 05 01:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133212 done Nov 05 01:26:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_502(8160937) WEXITSTATUS 0 Nov 05 01:26:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_502(8160937) done Nov 05 01:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_465(8160824) WEXITSTATUS 0 Nov 05 01:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_465(8160824) done Nov 05 01:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_987(8161656) WEXITSTATUS 0 Nov 05 01:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_987(8161656) done Nov 05 01:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_443(8160802) WEXITSTATUS 0 Nov 05 01:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_443(8160802) done Nov 05 01:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_369(8160662) WEXITSTATUS 0 Nov 05 01:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_369(8160662) done Nov 05 01:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1310(8135368) WEXITSTATUS 0 Nov 05 01:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1310(8135368) done Nov 05 01:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_984(8161653) WEXITSTATUS 0 Nov 05 01:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_984(8161653) done Nov 05 01:26:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162809 in pi_balou on r911u09n01 Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_439(8160798) WEXITSTATUS 0 Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_439(8160798) done Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_476(8160835) WEXITSTATUS 0 Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_476(8160835) done Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162654 in day on r913u01n04 Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162656 in day on r918u09n01 Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162661 in day on r815u09n07 Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162662 in day on r816u03n06 Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162663 in day on r816u09n03 Nov 05 01:26:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162664 in day on r816u15n07 Nov 05 01:26:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3657(8162810) in gpu on r808u11n01 Nov 05 01:26:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3658(8162811) in gpu on r4513u23n01 Nov 05 01:26:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5822(8162784) WEXITSTATUS 1 Nov 05 01:26:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5822(8162784) done Nov 05 01:26:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_257(8114523) WEXITSTATUS 0 Nov 05 01:26:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_257(8114523) done Nov 05 01:26:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1888(8162812) in day on r815u15n02 Nov 05 01:26:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_374(8160667) WEXITSTATUS 0 Nov 05 01:26:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_374(8160667) done Nov 05 01:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_683(8161194) WEXITSTATUS 0 Nov 05 01:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_683(8161194) done Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5842(8162813) in scavenge on r801u30n02 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5843(8162814) in scavenge on r801u27n02 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5844(8162815) in scavenge on r815u15n11 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5845(8162816) in scavenge on r806u02n03 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5846(8162817) in scavenge on r805u04n03 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5847(8162818) in scavenge on r805u14n04 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5848(8162819) in scavenge on r807u05n01 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5849(8162820) in scavenge on r807u11n04 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5850(8162821) in scavenge on r912u28n04 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5851(8162822) in scavenge on r912u32n03 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5852(8162823) in scavenge on r912u36n02 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5853(8162824) in scavenge on r912u36n02 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5854(8162825) in scavenge on r913u28n04 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5855(8162826) in scavenge on r815u15n07 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5856(8162827) in scavenge on r816u35n05 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5857(8162828) in scavenge on r815u23n01 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5858(8162829) in scavenge on r815u23n01 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5859(8162830) in scavenge on r815u23n01 Nov 05 01:26:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5860(8162831) in scavenge on r815u29n03 Nov 05 01:26:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5793(8162752) WEXITSTATUS 1 Nov 05 01:26:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5793(8162752) done Nov 05 01:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_494(8160853) WEXITSTATUS 0 Nov 05 01:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_494(8160853) done Nov 05 01:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_596(8161031) WEXITSTATUS 0 Nov 05 01:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_596(8161031) done Nov 05 01:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_356(8160649) WEXITSTATUS 0 Nov 05 01:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_356(8160649) done Nov 05 01:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_360(8160653) WEXITSTATUS 0 Nov 05 01:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_360(8160653) done Nov 05 01:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_477(8160836) WEXITSTATUS 0 Nov 05 01:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_477(8160836) done Nov 05 01:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_463(8160822) WEXITSTATUS 0 Nov 05 01:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_463(8160822) done Nov 05 01:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1347(8135492) WEXITSTATUS 0 Nov 05 01:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1347(8135492) done Nov 05 01:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_409(8160768) WEXITSTATUS 0 Nov 05 01:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_409(8160768) done Nov 05 01:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_464(8160823) WEXITSTATUS 0 Nov 05 01:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_464(8160823) done Nov 05 01:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_986(8161655) WEXITSTATUS 0 Nov 05 01:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_986(8161655) done Nov 05 01:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5310(8158772) WEXITSTATUS 1 Nov 05 01:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5310(8158772) done Nov 05 01:27:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_991(8161660) WEXITSTATUS 0 Nov 05 01:27:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_991(8161660) done Nov 05 01:27:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_492(8160851) WEXITSTATUS 0 Nov 05 01:27:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_492(8160851) done Nov 05 01:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_997(8161666) WEXITSTATUS 0 Nov 05 01:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_997(8161666) done Nov 05 01:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_474(8160833) WEXITSTATUS 0 Nov 05 01:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_474(8160833) done Nov 05 01:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_473(8160832) WEXITSTATUS 0 Nov 05 01:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_473(8160832) done Nov 05 01:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_659(8161170) WEXITSTATUS 0 Nov 05 01:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_659(8161170) done Nov 05 01:27:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_468(8160827) WEXITSTATUS 0 Nov 05 01:27:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_468(8160827) done Nov 05 01:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5091(8158415) WEXITSTATUS 1 Nov 05 01:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5091(8158415) done Nov 05 01:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_976(8161645) WEXITSTATUS 0 Nov 05 01:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_976(8161645) done Nov 05 01:27:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5858(8162829) WEXITSTATUS 1 Nov 05 01:27:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5858(8162829) done Nov 05 01:27:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_692(8161203) WEXITSTATUS 0 Nov 05 01:27:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_692(8161203) done Nov 05 01:27:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_466(8160825) WEXITSTATUS 0 Nov 05 01:27:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_466(8160825) done Nov 05 01:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_449(8160808) WEXITSTATUS 0 Nov 05 01:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_449(8160808) done Nov 05 01:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_371(8160664) WEXITSTATUS 0 Nov 05 01:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_371(8160664) done Nov 05 01:27:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_470(8160829) WEXITSTATUS 0 Nov 05 01:27:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_470(8160829) done Nov 05 01:27:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_662(8161173) WEXITSTATUS 0 Nov 05 01:27:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_662(8161173) done Nov 05 01:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157710 WEXITSTATUS 0 Nov 05 01:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157710 done Nov 05 01:27:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_438(8160797) WEXITSTATUS 0 Nov 05 01:27:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_438(8160797) done Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:27:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_429(8160788) WEXITSTATUS 0 Nov 05 01:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_429(8160788) done Nov 05 01:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_499(8160858) WEXITSTATUS 0 Nov 05 01:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_499(8160858) done Nov 05 01:27:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_684(8161195) WEXITSTATUS 0 Nov 05 01:27:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_684(8161195) done Nov 05 01:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_635(8161146) WEXITSTATUS 0 Nov 05 01:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_635(8161146) done Nov 05 01:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_417(8160776) WEXITSTATUS 0 Nov 05 01:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_417(8160776) done Nov 05 01:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_697(8161208) WEXITSTATUS 0 Nov 05 01:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_697(8161208) done Nov 05 01:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_516(8160951) WEXITSTATUS 0 Nov 05 01:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_516(8160951) done Nov 05 01:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_471(8160830) WEXITSTATUS 0 Nov 05 01:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_471(8160830) done Nov 05 01:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5844(8162815) WEXITSTATUS 1 Nov 05 01:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5844(8162815) done Nov 05 01:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_517(8160952) WEXITSTATUS 0 Nov 05 01:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_517(8160952) done Nov 05 01:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_669(8161180) WEXITSTATUS 0 Nov 05 01:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_669(8161180) done Nov 05 01:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_461(8160820) WEXITSTATUS 0 Nov 05 01:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_461(8160820) done Nov 05 01:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_583(8161018) WEXITSTATUS 0 Nov 05 01:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_583(8161018) done Nov 05 01:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_663(8161174) WEXITSTATUS 0 Nov 05 01:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_663(8161174) done Nov 05 01:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_994(8161663) WEXITSTATUS 0 Nov 05 01:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_994(8161663) done Nov 05 01:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_673(8161184) WEXITSTATUS 0 Nov 05 01:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_673(8161184) done Nov 05 01:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_695(8161206) WEXITSTATUS 0 Nov 05 01:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_695(8161206) done Nov 05 01:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_992(8161661) WEXITSTATUS 0 Nov 05 01:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_992(8161661) done Nov 05 01:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_357(8160650) WEXITSTATUS 0 Nov 05 01:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_357(8160650) done Nov 05 01:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_448(8160807) WEXITSTATUS 0 Nov 05 01:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_448(8160807) done Nov 05 01:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5435(8159559) WEXITSTATUS 1 Nov 05 01:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5435(8159559) done Nov 05 01:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_557(8160992) WEXITSTATUS 0 Nov 05 01:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_557(8160992) done Nov 05 01:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_989(8161658) WEXITSTATUS 0 Nov 05 01:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_989(8161658) done Nov 05 01:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157709 WEXITSTATUS 0 Nov 05 01:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157709 done Nov 05 01:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_649(8161160) WEXITSTATUS 0 Nov 05 01:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_649(8161160) done Nov 05 01:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_682(8161193) WEXITSTATUS 0 Nov 05 01:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_682(8161193) done Nov 05 01:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_428(8160787) WEXITSTATUS 0 Nov 05 01:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_428(8160787) done Nov 05 01:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154608 WEXITSTATUS 0 Nov 05 01:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154608 done Nov 05 01:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_437(8160796) WEXITSTATUS 0 Nov 05 01:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_437(8160796) done Nov 05 01:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_412(8160771) WEXITSTATUS 0 Nov 05 01:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_412(8160771) done Nov 05 01:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_654(8161165) WEXITSTATUS 0 Nov 05 01:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_654(8161165) done Nov 05 01:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_661(8161172) WEXITSTATUS 0 Nov 05 01:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_661(8161172) done Nov 05 01:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_472(8160831) WEXITSTATUS 0 Nov 05 01:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_472(8160831) done Nov 05 01:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_447(8160806) WEXITSTATUS 0 Nov 05 01:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_447(8160806) done Nov 05 01:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_457(8160816) WEXITSTATUS 0 Nov 05 01:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_457(8160816) done Nov 05 01:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_442(8160801) WEXITSTATUS 0 Nov 05 01:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_442(8160801) done Nov 05 01:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_658(8161169) WEXITSTATUS 0 Nov 05 01:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_658(8161169) done Nov 05 01:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_998(8161667) WEXITSTATUS 0 Nov 05 01:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_998(8161667) done Nov 05 01:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_396(8160689) WEXITSTATUS 0 Nov 05 01:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_396(8160689) done Nov 05 01:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_560(8160995) WEXITSTATUS 0 Nov 05 01:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_560(8160995) done Nov 05 01:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_670(8161181) WEXITSTATUS 0 Nov 05 01:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_670(8161181) done Nov 05 01:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_421(8160780) WEXITSTATUS 0 Nov 05 01:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_421(8160780) done Nov 05 01:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_462(8160821) WEXITSTATUS 0 Nov 05 01:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_462(8160821) done Nov 05 01:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_985(8161654) WEXITSTATUS 0 Nov 05 01:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_985(8161654) done Nov 05 01:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_458(8160817) WEXITSTATUS 0 Nov 05 01:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_458(8160817) done Nov 05 01:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_542(8160977) WEXITSTATUS 0 Nov 05 01:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_542(8160977) done Nov 05 01:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_415(8160774) WEXITSTATUS 0 Nov 05 01:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_415(8160774) done Nov 05 01:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_498(8160857) WEXITSTATUS 0 Nov 05 01:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_498(8160857) done Nov 05 01:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_689(8161200) WEXITSTATUS 0 Nov 05 01:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_689(8161200) done Nov 05 01:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_990(8161659) WEXITSTATUS 0 Nov 05 01:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_990(8161659) done Nov 05 01:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_407(8160766) WEXITSTATUS 0 Nov 05 01:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_407(8160766) done Nov 05 01:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_431(8160790) WEXITSTATUS 0 Nov 05 01:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_431(8160790) done Nov 05 01:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_424(8160783) WEXITSTATUS 0 Nov 05 01:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_424(8160783) done Nov 05 01:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_694(8161205) WEXITSTATUS 0 Nov 05 01:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_694(8161205) done Nov 05 01:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5672(8162164) WEXITSTATUS 1 Nov 05 01:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5672(8162164) done Nov 05 01:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_480(8160839) WEXITSTATUS 0 Nov 05 01:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_480(8160839) done Nov 05 01:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_485(8160844) WEXITSTATUS 0 Nov 05 01:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_485(8160844) done Nov 05 01:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_483(8160842) WEXITSTATUS 0 Nov 05 01:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_483(8160842) done Nov 05 01:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_441(8160800) WEXITSTATUS 0 Nov 05 01:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_441(8160800) done Nov 05 01:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_675(8161186) WEXITSTATUS 0 Nov 05 01:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_675(8161186) done Nov 05 01:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_664(8161175) WEXITSTATUS 0 Nov 05 01:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_664(8161175) done Nov 05 01:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_420(8160779) WEXITSTATUS 0 Nov 05 01:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_420(8160779) done Nov 05 01:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_490(8160849) WEXITSTATUS 0 Nov 05 01:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_490(8160849) done Nov 05 01:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_671(8161182) WEXITSTATUS 0 Nov 05 01:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_671(8161182) done Nov 05 01:28:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_434(8160793) WEXITSTATUS 0 Nov 05 01:28:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_434(8160793) done Nov 05 01:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_459(8160818) WEXITSTATUS 0 Nov 05 01:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_459(8160818) done Nov 05 01:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_633(8161144) WEXITSTATUS 0 Nov 05 01:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_633(8161144) done Nov 05 01:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_393(8160686) WEXITSTATUS 0 Nov 05 01:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_393(8160686) done Nov 05 01:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_392(8160685) WEXITSTATUS 0 Nov 05 01:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_392(8160685) done Nov 05 01:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_469(8160828) WEXITSTATUS 0 Nov 05 01:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_469(8160828) done Nov 05 01:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_373(8160666) WEXITSTATUS 0 Nov 05 01:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_373(8160666) done Nov 05 01:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_427(8160786) WEXITSTATUS 0 Nov 05 01:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_427(8160786) done Nov 05 01:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_686(8161197) WEXITSTATUS 0 Nov 05 01:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_686(8161197) done Nov 05 01:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_444(8160803) WEXITSTATUS 0 Nov 05 01:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_444(8160803) done Nov 05 01:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5583(8161403) WEXITSTATUS 1 Nov 05 01:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5583(8161403) done Nov 05 01:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_687(8161198) WEXITSTATUS 0 Nov 05 01:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_687(8161198) done Nov 05 01:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_406(8160765) WEXITSTATUS 0 Nov 05 01:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_406(8160765) done Nov 05 01:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_681(8161192) WEXITSTATUS 0 Nov 05 01:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_681(8161192) done Nov 05 01:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_423(8160782) WEXITSTATUS 0 Nov 05 01:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_423(8160782) done Nov 05 01:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_660(8161171) WEXITSTATUS 0 Nov 05 01:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_660(8161171) done Nov 05 01:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_440(8160799) WEXITSTATUS 0 Nov 05 01:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_440(8160799) done Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_665(8161176) WEXITSTATUS 0 Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_665(8161176) done Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_666(8161177) WEXITSTATUS 0 Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_666(8161177) done Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_667(8161178) WEXITSTATUS 0 Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_667(8161178) done Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_398(8160691) WEXITSTATUS 0 Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_398(8160691) done Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_484(8160843) WEXITSTATUS 0 Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_484(8160843) done Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5774(8162716) WEXITSTATUS 1 Nov 05 01:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5774(8162716) done Nov 05 01:28:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1400 Nov 05 01:28:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_481(8160840) WEXITSTATUS 0 Nov 05 01:28:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_481(8160840) done Nov 05 01:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_981(8161650) WEXITSTATUS 0 Nov 05 01:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_981(8161650) done Nov 05 01:28:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_680(8161191) WEXITSTATUS 0 Nov 05 01:28:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_680(8161191) done Nov 05 01:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_999(8159941) WEXITSTATUS 0 Nov 05 01:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_999(8159941) done Nov 05 01:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_495(8160854) WEXITSTATUS 0 Nov 05 01:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_495(8160854) done Nov 05 01:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_525(8160960) WEXITSTATUS 0 Nov 05 01:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_525(8160960) done Nov 05 01:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_585(8161020) WEXITSTATUS 0 Nov 05 01:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_585(8161020) done Nov 05 01:29:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_581(8161016) WEXITSTATUS 0 Nov 05 01:29:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_581(8161016) done Nov 05 01:29:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_395(8160688) WEXITSTATUS 0 Nov 05 01:29:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_395(8160688) done Nov 05 01:29:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_657(8161168) WEXITSTATUS 0 Nov 05 01:29:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_657(8161168) done Nov 05 01:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_582(8161017) WEXITSTATUS 0 Nov 05 01:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_582(8161017) done Nov 05 01:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_679(8161190) WEXITSTATUS 0 Nov 05 01:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_679(8161190) done Nov 05 01:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_123(8160249) WEXITSTATUS 0 Nov 05 01:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_123(8160249) done Nov 05 01:29:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_400(8160759) WEXITSTATUS 0 Nov 05 01:29:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_400(8160759) done Nov 05 01:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_651(8161162) WEXITSTATUS 0 Nov 05 01:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_651(8161162) done Nov 05 01:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_584(8161019) WEXITSTATUS 0 Nov 05 01:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_584(8161019) done Nov 05 01:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_620(8161131) WEXITSTATUS 0 Nov 05 01:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_620(8161131) done Nov 05 01:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_685(8161196) WEXITSTATUS 0 Nov 05 01:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_685(8161196) done Nov 05 01:29:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_637(8161148) WEXITSTATUS 0 Nov 05 01:29:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_637(8161148) done Nov 05 01:29:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_515(8160950) WEXITSTATUS 0 Nov 05 01:29:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_515(8160950) done Nov 05 01:29:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_548(8160983) WEXITSTATUS 0 Nov 05 01:29:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_548(8160983) done Nov 05 01:29:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_648(8161159) WEXITSTATUS 0 Nov 05 01:29:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_648(8161159) done Nov 05 01:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157719 WEXITSTATUS 0 Nov 05 01:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157719 done Nov 05 01:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_696(8161207) WEXITSTATUS 0 Nov 05 01:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_696(8161207) done Nov 05 01:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_993(8161662) WEXITSTATUS 0 Nov 05 01:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_993(8161662) done Nov 05 01:29:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_768(8161363) WEXITSTATUS 0 Nov 05 01:29:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_768(8161363) done Nov 05 01:29:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_539(8160974) WEXITSTATUS 0 Nov 05 01:29:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_539(8160974) done Nov 05 01:29:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_533(8160968) WEXITSTATUS 0 Nov 05 01:29:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_533(8160968) done Nov 05 01:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157703 WEXITSTATUS 0 Nov 05 01:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157703 done Nov 05 01:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_713(8161308) WEXITSTATUS 0 Nov 05 01:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_713(8161308) done Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:29:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_532(8160967) WEXITSTATUS 0 Nov 05 01:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_532(8160967) done Nov 05 01:29:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_580(8161015) WEXITSTATUS 0 Nov 05 01:29:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_580(8161015) done Nov 05 01:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_337(8160630) WEXITSTATUS 0 Nov 05 01:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_337(8160630) done Nov 05 01:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_419(8160778) WEXITSTATUS 0 Nov 05 01:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_419(8160778) done Nov 05 01:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_653(8161164) WEXITSTATUS 0 Nov 05 01:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_653(8161164) done Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_693(8161204) WEXITSTATUS 0 Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_693(8161204) done Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_509(8160944) WEXITSTATUS 0 Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_509(8160944) done Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_528(8160963) WEXITSTATUS 0 Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_528(8160963) done Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_488(8160847) WEXITSTATUS 0 Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_488(8160847) done Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_676(8161187) WEXITSTATUS 0 Nov 05 01:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_676(8161187) done Nov 05 01:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_698(8161209) WEXITSTATUS 0 Nov 05 01:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_698(8161209) done Nov 05 01:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_674(8161185) WEXITSTATUS 0 Nov 05 01:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_674(8161185) done Nov 05 01:29:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154657 in day on r806u23n01 Nov 05 01:29:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154662 in day on r816u15n05 Nov 05 01:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_624(8161135) WEXITSTATUS 0 Nov 05 01:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_624(8161135) done Nov 05 01:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3657(8162810) WEXITSTATUS 0 Nov 05 01:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3657(8162810) done Nov 05 01:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_672(8161183) WEXITSTATUS 0 Nov 05 01:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_672(8161183) done Nov 05 01:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_430(8160789) WEXITSTATUS 0 Nov 05 01:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_430(8160789) done Nov 05 01:29:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 01:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_491(8160850) WEXITSTATUS 0 Nov 05 01:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_491(8160850) done Nov 05 01:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_621(8161132) WEXITSTATUS 0 Nov 05 01:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_621(8161132) done Nov 05 01:29:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162832 InitPrio=21238 usec=3477 Nov 05 01:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_397(8160690) WEXITSTATUS 0 Nov 05 01:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_397(8160690) done Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5861(8162833) in scavenge on r907u05n04 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5862(8162834) in scavenge on r802u06n01 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5863(8162835) in scavenge on r917u01n04 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5864(8162836) in scavenge on r815u03n09 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5865(8162837) in scavenge on r815u15n10 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5866(8162838) in scavenge on r815u15n11 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5867(8162839) in scavenge on r816u23n08 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5868(8162840) in scavenge on r816u35n10 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5869(8162841) in scavenge on r816u35n12 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5870(8162842) in scavenge on r913u05n01 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5871(8162843) in scavenge on r815u09n08 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5872(8162844) in scavenge on r815u09n08 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5873(8162845) in scavenge on r816u23n04 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5874(8162846) in scavenge on r815u23n01 Nov 05 01:29:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5875(8162847) in scavenge on r815u35n05 Nov 05 01:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_677(8161188) WEXITSTATUS 0 Nov 05 01:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_677(8161188) done Nov 05 01:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_575(8161010) WEXITSTATUS 0 Nov 05 01:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_575(8161010) done Nov 05 01:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_425(8160784) WEXITSTATUS 0 Nov 05 01:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_425(8160784) done Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_436(8160795) WEXITSTATUS 0 Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_436(8160795) done Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157714 WEXITSTATUS 0 Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157714 done Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_574(8161009) WEXITSTATUS 0 Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_574(8161009) done Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5740(8162596) WEXITSTATUS 1 Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5740(8162596) done Nov 05 01:29:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1889(8162848) in scavenge on r816u23n08 Nov 05 01:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_546(8160981) WEXITSTATUS 0 Nov 05 01:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_546(8160981) done Nov 05 01:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_656(8161167) WEXITSTATUS 0 Nov 05 01:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_656(8161167) done Nov 05 01:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_650(8161161) WEXITSTATUS 0 Nov 05 01:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_650(8161161) done Nov 05 01:29:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_487(8160846) WEXITSTATUS 0 Nov 05 01:29:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_487(8160846) done Nov 05 01:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_550(8160985) WEXITSTATUS 0 Nov 05 01:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_550(8160985) done Nov 05 01:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4742(8156982) WEXITSTATUS 1 Nov 05 01:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4742(8156982) done Nov 05 01:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_647(8161158) WEXITSTATUS 0 Nov 05 01:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_647(8161158) done Nov 05 01:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_527(8160962) WEXITSTATUS 0 Nov 05 01:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_527(8160962) done Nov 05 01:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_543(8160978) WEXITSTATUS 0 Nov 05 01:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_543(8160978) done Nov 05 01:30:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5612(8161713) WEXITSTATUS 1 Nov 05 01:30:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5612(8161713) done Nov 05 01:30:03 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8157720 had to be killed Nov 05 01:30:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_416(8160775) WEXITSTATUS 0 Nov 05 01:30:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_416(8160775) done Nov 05 01:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_422(8160781) WEXITSTATUS 0 Nov 05 01:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_422(8160781) done Nov 05 01:30:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_577(8161012) WEXITSTATUS 0 Nov 05 01:30:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_577(8161012) done Nov 05 01:30:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_545(8160980) WEXITSTATUS 0 Nov 05 01:30:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_545(8160980) done Nov 05 01:30:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_668(8161179) WEXITSTATUS 0 Nov 05 01:30:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_668(8161179) done Nov 05 01:30:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_500(8160935) WEXITSTATUS 0 Nov 05 01:30:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_500(8160935) done Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_623(8161134) WEXITSTATUS 0 Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_623(8161134) done Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_510(8160945) WEXITSTATUS 0 Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_510(8160945) done Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5809(8162769) WEXITSTATUS 1 Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5809(8162769) done Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_690(8161201) WEXITSTATUS 0 Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_690(8161201) done Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_619(8161130) WEXITSTATUS 0 Nov 05 01:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_619(8161130) done Nov 05 01:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_489(8160848) WEXITSTATUS 0 Nov 05 01:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_489(8160848) done Nov 05 01:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_556(8160991) WEXITSTATUS 0 Nov 05 01:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_556(8160991) done Nov 05 01:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_652(8161163) WEXITSTATUS 0 Nov 05 01:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_652(8161163) done Nov 05 01:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_508(8160943) WEXITSTATUS 0 Nov 05 01:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_508(8160943) done Nov 05 01:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_535(8160970) WEXITSTATUS 0 Nov 05 01:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_535(8160970) done Nov 05 01:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_521(8160956) WEXITSTATUS 0 Nov 05 01:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_521(8160956) done Nov 05 01:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_734(8161329) WEXITSTATUS 0 Nov 05 01:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_734(8161329) done Nov 05 01:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_538(8160973) WEXITSTATUS 0 Nov 05 01:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_538(8160973) done Nov 05 01:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5658(8162025) WEXITSTATUS 1 Nov 05 01:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5658(8162025) done Nov 05 01:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_507(8160942) WEXITSTATUS 0 Nov 05 01:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_507(8160942) done Nov 05 01:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_426(8160785) WEXITSTATUS 0 Nov 05 01:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_426(8160785) done Nov 05 01:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_578(8161013) WEXITSTATUS 0 Nov 05 01:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_578(8161013) done Nov 05 01:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_524(8160959) WEXITSTATUS 0 Nov 05 01:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_524(8160959) done Nov 05 01:30:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_773(8161368) WEXITSTATUS 0 Nov 05 01:30:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_773(8161368) done Nov 05 01:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_759(8161354) WEXITSTATUS 0 Nov 05 01:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_759(8161354) done Nov 05 01:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_530(8160965) WEXITSTATUS 0 Nov 05 01:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_530(8160965) done Nov 05 01:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_551(8160986) WEXITSTATUS 0 Nov 05 01:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_551(8160986) done Nov 05 01:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_413(8160772) WEXITSTATUS 0 Nov 05 01:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_413(8160772) done Nov 05 01:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_537(8160972) WEXITSTATUS 0 Nov 05 01:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_537(8160972) done Nov 05 01:30:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_520(8160955) WEXITSTATUS 0 Nov 05 01:30:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_520(8160955) done Nov 05 01:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_688(8161199) WEXITSTATUS 0 Nov 05 01:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_688(8161199) done Nov 05 01:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_501(8160936) WEXITSTATUS 0 Nov 05 01:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_501(8160936) done Nov 05 01:30:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_762(8161357) WEXITSTATUS 0 Nov 05 01:30:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_762(8161357) done Nov 05 01:30:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_678(8161189) WEXITSTATUS 0 Nov 05 01:30:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_678(8161189) done Nov 05 01:30:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_576(8161011) WEXITSTATUS 0 Nov 05 01:30:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_576(8161011) done Nov 05 01:30:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_570(8161005) WEXITSTATUS 0 Nov 05 01:30:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_570(8161005) done Nov 05 01:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_579(8161014) WEXITSTATUS 0 Nov 05 01:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_579(8161014) done Nov 05 01:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_479(8160838) WEXITSTATUS 0 Nov 05 01:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_479(8160838) done Nov 05 01:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_496(8160855) WEXITSTATUS 0 Nov 05 01:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_496(8160855) done Nov 05 01:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_531(8160966) WEXITSTATUS 0 Nov 05 01:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_531(8160966) done Nov 05 01:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_592(8161027) WEXITSTATUS 0 Nov 05 01:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_592(8161027) done Nov 05 01:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_736(8161331) WEXITSTATUS 0 Nov 05 01:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_736(8161331) done Nov 05 01:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_717(8161312) WEXITSTATUS 0 Nov 05 01:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_717(8161312) done Nov 05 01:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_522(8160957) WEXITSTATUS 0 Nov 05 01:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_522(8160957) done Nov 05 01:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157717 WEXITSTATUS 0 Nov 05 01:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157717 done Nov 05 01:30:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_590(8161025) WEXITSTATUS 0 Nov 05 01:30:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_590(8161025) done Nov 05 01:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_772(8161367) WEXITSTATUS 0 Nov 05 01:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_772(8161367) done Nov 05 01:30:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_514(8160949) WEXITSTATUS 0 Nov 05 01:30:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_514(8160949) done Nov 05 01:30:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_534(8160969) WEXITSTATUS 0 Nov 05 01:30:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_534(8160969) done Nov 05 01:30:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_589(8161024) WEXITSTATUS 0 Nov 05 01:30:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_589(8161024) done Nov 05 01:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157708 WEXITSTATUS 0 Nov 05 01:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157708 done Nov 05 01:30:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_512(8160947) WEXITSTATUS 0 Nov 05 01:30:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_512(8160947) done Nov 05 01:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3632(8162353) WEXITSTATUS 0 Nov 05 01:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3632(8162353) done Nov 05 01:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_486(8160845) WEXITSTATUS 0 Nov 05 01:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_486(8160845) done Nov 05 01:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_482(8160841) WEXITSTATUS 0 Nov 05 01:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_482(8160841) done Nov 05 01:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_541(8160976) WEXITSTATUS 0 Nov 05 01:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_541(8160976) done Nov 05 01:30:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_568(8161003) WEXITSTATUS 0 Nov 05 01:30:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_568(8161003) done Nov 05 01:30:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154606 WEXITSTATUS 0 Nov 05 01:30:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154606 done Nov 05 01:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_394(8160687) WEXITSTATUS 0 Nov 05 01:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_394(8160687) done Nov 05 01:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_529(8160964) WEXITSTATUS 0 Nov 05 01:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_529(8160964) done Nov 05 01:31:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_51(8160074) WEXITSTATUS 0 Nov 05 01:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_51(8160074) done Nov 05 01:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_712(8161307) WEXITSTATUS 0 Nov 05 01:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_712(8161307) done Nov 05 01:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_715(8161310) WEXITSTATUS 0 Nov 05 01:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_715(8161310) done Nov 05 01:31:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_564(8160999) WEXITSTATUS 0 Nov 05 01:31:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_564(8160999) done Nov 05 01:31:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5655(8161948) WEXITSTATUS 1 Nov 05 01:31:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5655(8161948) done Nov 05 01:31:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5279(8158721) WEXITSTATUS 1 Nov 05 01:31:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5279(8158721) done Nov 05 01:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_547(8160982) WEXITSTATUS 0 Nov 05 01:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_547(8160982) done Nov 05 01:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_978(8161647) WEXITSTATUS 0 Nov 05 01:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_978(8161647) done Nov 05 01:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_766(8161361) WEXITSTATUS 0 Nov 05 01:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_766(8161361) done Nov 05 01:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159912 WEXITSTATUS 0 Nov 05 01:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159912 done Nov 05 01:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_558(8160993) WEXITSTATUS 0 Nov 05 01:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_558(8160993) done Nov 05 01:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1412(8135945) WEXITSTATUS 0 Nov 05 01:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1412(8135945) done Nov 05 01:31:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_599(8161034) WEXITSTATUS 0 Nov 05 01:31:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_599(8161034) done Nov 05 01:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_634(8161145) WEXITSTATUS 0 Nov 05 01:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_634(8161145) done Nov 05 01:31:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_526(8160961) WEXITSTATUS 0 Nov 05 01:31:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_526(8160961) done Nov 05 01:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1448(8136423) WEXITSTATUS 0 Nov 05 01:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1448(8136423) done Nov 05 01:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_761(8161356) WEXITSTATUS 0 Nov 05 01:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_761(8161356) done Nov 05 01:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_735(8161330) WEXITSTATUS 0 Nov 05 01:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_735(8161330) done Nov 05 01:31:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 01:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_591(8161026) WEXITSTATUS 0 Nov 05 01:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_591(8161026) done Nov 05 01:31:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_414(8160773) WEXITSTATUS 0 Nov 05 01:31:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_414(8160773) done Nov 05 01:31:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_636(8161147) WEXITSTATUS 0 Nov 05 01:31:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_636(8161147) done Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:31:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:31:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_631(8161142) WEXITSTATUS 0 Nov 05 01:31:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_631(8161142) done Nov 05 01:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_763(8161358) WEXITSTATUS 0 Nov 05 01:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_763(8161358) done Nov 05 01:31:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_418(8160777) WEXITSTATUS 0 Nov 05 01:31:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_418(8160777) done Nov 05 01:31:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_559(8160994) WEXITSTATUS 0 Nov 05 01:31:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_559(8160994) done Nov 05 01:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5837(8162804) WEXITSTATUS 1 Nov 05 01:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5837(8162804) done Nov 05 01:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1146(8123261) WEXITSTATUS 1 Nov 05 01:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1146(8123261) done Nov 05 01:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5447(8159763) WEXITSTATUS 1 Nov 05 01:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5447(8159763) done Nov 05 01:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_769(8161364) WEXITSTATUS 0 Nov 05 01:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_769(8161364) done Nov 05 01:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5601(8161688) WEXITSTATUS 1 Nov 05 01:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5601(8161688) done Nov 05 01:31:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5578(8161232) WEXITSTATUS 1 Nov 05 01:31:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5578(8161232) done Nov 05 01:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_549(8160984) WEXITSTATUS 0 Nov 05 01:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_549(8160984) done Nov 05 01:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154656 WEXITSTATUS 0 Nov 05 01:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154656 done Nov 05 01:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_767(8161362) WEXITSTATUS 0 Nov 05 01:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_767(8161362) done Nov 05 01:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_563(8160998) WEXITSTATUS 0 Nov 05 01:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_563(8160998) done Nov 05 01:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_737(8161332) WEXITSTATUS 0 Nov 05 01:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_737(8161332) done Nov 05 01:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_757(8161352) WEXITSTATUS 0 Nov 05 01:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_757(8161352) done Nov 05 01:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_699(8161210) WEXITSTATUS 0 Nov 05 01:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_699(8161210) done Nov 05 01:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_655(8161166) WEXITSTATUS 0 Nov 05 01:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_655(8161166) done Nov 05 01:31:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_595(8161030) WEXITSTATUS 0 Nov 05 01:31:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_595(8161030) done Nov 05 01:31:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157718 WEXITSTATUS 0 Nov 05 01:31:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157718 done Nov 05 01:31:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_544(8160979) WEXITSTATUS 0 Nov 05 01:31:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_544(8160979) done Nov 05 01:31:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_594(8161029) WEXITSTATUS 0 Nov 05 01:31:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_594(8161029) done Nov 05 01:31:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_588(8161023) WEXITSTATUS 0 Nov 05 01:31:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_588(8161023) done Nov 05 01:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_760(8161355) WEXITSTATUS 0 Nov 05 01:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_760(8161355) done Nov 05 01:31:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_554(8160989) WEXITSTATUS 0 Nov 05 01:31:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_554(8160989) done Nov 05 01:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_562(8160997) WEXITSTATUS 0 Nov 05 01:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_562(8160997) done Nov 05 01:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_565(8161000) WEXITSTATUS 0 Nov 05 01:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_565(8161000) done Nov 05 01:32:03 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2163(8125996) Nov 05 01:32:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157722 WEXITSTATUS 0 Nov 05 01:32:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157722 done Nov 05 01:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_630(8161141) WEXITSTATUS 0 Nov 05 01:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_630(8161141) done Nov 05 01:32:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4811(8157768) WEXITSTATUS 1 Nov 05 01:32:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4811(8157768) done Nov 05 01:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_593(8161028) WEXITSTATUS 0 Nov 05 01:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_593(8161028) done Nov 05 01:32:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5807(8162767) WEXITSTATUS 1 Nov 05 01:32:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5807(8162767) done Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_553(8160988) WEXITSTATUS 0 Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_553(8160988) done Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_445(8160804) WEXITSTATUS 0 Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_445(8160804) done Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_109(8160235) WEXITSTATUS 0 Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_109(8160235) done Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n03 Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162659 in day on r805u06n01 Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3659(8162849) in gpu on r808u11n01 Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3660(8162850) in gpu on r903u34n01 Nov 05 01:32:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154660 in day on r919u05n02 Nov 05 01:32:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_504(8160939) WEXITSTATUS 0 Nov 05 01:32:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_504(8160939) done Nov 05 01:32:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1890(8162851) in day on r816u09n04 Nov 05 01:32:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_569(8161004) WEXITSTATUS 0 Nov 05 01:32:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_569(8161004) done Nov 05 01:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_764(8161359) WEXITSTATUS 0 Nov 05 01:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_764(8161359) done Nov 05 01:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_566(8161001) WEXITSTATUS 0 Nov 05 01:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_566(8161001) done Nov 05 01:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_771(8161366) WEXITSTATUS 0 Nov 05 01:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_771(8161366) done Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5876(8162852) in scavenge on r801u04n03 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5877(8162853) in scavenge on r903u04n03 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5878(8162854) in scavenge on r903u04n03 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5879(8162855) in scavenge on r815u09n12 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5880(8162856) in scavenge on r815u09n12 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5881(8162857) in scavenge on r816u23n08 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5882(8162858) in scavenge on r816u35n12 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5883(8162859) in scavenge on r815u23n07 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5884(8162860) in scavenge on r815u23n09 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5885(8162861) in scavenge on r815u23n09 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5886(8162862) in scavenge on r815u23n12 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5887(8162863) in scavenge on r815u29n11 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5888(8162864) in scavenge on r808u14n01 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5889(8162865) in scavenge on r913u13n03 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5890(8162866) in scavenge on r914u01n01 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5891(8162867) in scavenge on r918u36n04 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5892(8162868) in scavenge on r805u04n02 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5893(8162869) in scavenge on r805u09n01 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5894(8162870) in scavenge on r805u09n04 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5895(8162871) in scavenge on r815u09n04 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5896(8162872) in scavenge on r816u03n02 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5897(8162873) in scavenge on r816u03n02 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5898(8162874) in scavenge on r816u09n02 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5899(8162875) in scavenge on r816u35n01 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5900(8162876) in scavenge on r816u35n06 Nov 05 01:32:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5901(8162877) in scavenge on r908u24n02 Nov 05 01:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_505(8160940) WEXITSTATUS 0 Nov 05 01:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_505(8160940) done Nov 05 01:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_754(8161349) WEXITSTATUS 0 Nov 05 01:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_754(8161349) done Nov 05 01:32:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_213(7572381) in scavenge on r815u09n04 Nov 05 01:32:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_663(8114932) in scavenge on r816u09n02 Nov 05 01:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157727 WEXITSTATUS 0 Nov 05 01:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157727 done Nov 05 01:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5888(8162864) WEXITSTATUS 1 Nov 05 01:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5888(8162864) done Nov 05 01:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_752(8161347) WEXITSTATUS 0 Nov 05 01:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_752(8161347) done Nov 05 01:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_523(8160958) WEXITSTATUS 0 Nov 05 01:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_523(8160958) done Nov 05 01:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_716(8161311) WEXITSTATUS 0 Nov 05 01:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_716(8161311) done Nov 05 01:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157721 WEXITSTATUS 0 Nov 05 01:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157721 done Nov 05 01:32:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_710(8161305) WEXITSTATUS 0 Nov 05 01:32:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_710(8161305) done Nov 05 01:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 01:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 01:32:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 01:32:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5885(8162861) WEXITSTATUS 1 Nov 05 01:32:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5885(8162861) done Nov 05 01:32:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_755(8161350) WEXITSTATUS 0 Nov 05 01:32:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_755(8161350) done Nov 05 01:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_758(8161353) WEXITSTATUS 0 Nov 05 01:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_758(8161353) done Nov 05 01:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5179(8158533) WEXITSTATUS 1 Nov 05 01:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5179(8158533) done Nov 05 01:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4051(8147292) WEXITSTATUS 1 Nov 05 01:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4051(8147292) done Nov 05 01:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_765(8161360) WEXITSTATUS 0 Nov 05 01:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_765(8161360) done Nov 05 01:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_751(8161346) WEXITSTATUS 0 Nov 05 01:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_751(8161346) done Nov 05 01:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157726 WEXITSTATUS 0 Nov 05 01:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157726 done Nov 05 01:32:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5648(8161941) WEXITSTATUS 1 Nov 05 01:32:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5648(8161941) done Nov 05 01:33:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_750(8161345) WEXITSTATUS 0 Nov 05 01:33:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_750(8161345) done Nov 05 01:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4698(8156470) WEXITSTATUS 1 Nov 05 01:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4698(8156470) done Nov 05 01:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_540(8160975) WEXITSTATUS 0 Nov 05 01:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_540(8160975) done Nov 05 01:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3645(8162711) WEXITSTATUS 0 Nov 05 01:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3645(8162711) done Nov 05 01:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_536(8160971) WEXITSTATUS 0 Nov 05 01:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_536(8160971) done Nov 05 01:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1488(8139218) WEXITSTATUS 0 Nov 05 01:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1488(8139218) done Nov 05 01:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_711(8161306) WEXITSTATUS 0 Nov 05 01:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_711(8161306) done Nov 05 01:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_756(8161351) WEXITSTATUS 0 Nov 05 01:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_756(8161351) done Nov 05 01:33:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_552(8160987) WEXITSTATUS 0 Nov 05 01:33:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_552(8160987) done Nov 05 01:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_456(8160815) WEXITSTATUS 0 Nov 05 01:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_456(8160815) done Nov 05 01:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_625(8161136) WEXITSTATUS 0 Nov 05 01:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_625(8161136) done Nov 05 01:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_626(8161137) WEXITSTATUS 0 Nov 05 01:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_626(8161137) done Nov 05 01:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_979(8161648) WEXITSTATUS 0 Nov 05 01:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_979(8161648) done Nov 05 01:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_454(8160813) WEXITSTATUS 0 Nov 05 01:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_454(8160813) done Nov 05 01:33:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_513(8160948) WEXITSTATUS 0 Nov 05 01:33:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_513(8160948) done Nov 05 01:33:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154624 WEXITSTATUS 0 Nov 05 01:33:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154624 done Nov 05 01:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_770(8161365) WEXITSTATUS 0 Nov 05 01:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_770(8161365) done Nov 05 01:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157728 WEXITSTATUS 0 Nov 05 01:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157728 done Nov 05 01:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_189(8160315) WEXITSTATUS 0 Nov 05 01:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_189(8160315) done Nov 05 01:33:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_632(8161143) WEXITSTATUS 0 Nov 05 01:33:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_632(8161143) done Nov 05 01:33:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_453(8160812) WEXITSTATUS 0 Nov 05 01:33:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_453(8160812) done Nov 05 01:33:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5691(8162313) WEXITSTATUS 1 Nov 05 01:33:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5691(8162313) done Nov 05 01:33:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5886(8162862) WEXITSTATUS 1 Nov 05 01:33:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5886(8162862) done Nov 05 01:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_743(8161338) WEXITSTATUS 0 Nov 05 01:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_743(8161338) done Nov 05 01:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_748(8161343) WEXITSTATUS 0 Nov 05 01:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_748(8161343) done Nov 05 01:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_744(8161339) WEXITSTATUS 0 Nov 05 01:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_744(8161339) done Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_980(8161649) WEXITSTATUS 0 Nov 05 01:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_980(8161649) done Nov 05 01:33:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_603(8161114) WEXITSTATUS 0 Nov 05 01:33:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_603(8161114) done Nov 05 01:33:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157730 WEXITSTATUS 0 Nov 05 01:33:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157730 done Nov 05 01:33:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_452(8160811) WEXITSTATUS 0 Nov 05 01:33:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_452(8160811) done Nov 05 01:33:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_753(8161348) WEXITSTATUS 0 Nov 05 01:33:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_753(8161348) done Nov 05 01:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_747(8161342) WEXITSTATUS 0 Nov 05 01:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_747(8161342) done Nov 05 01:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_741(8161336) WEXITSTATUS 0 Nov 05 01:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_741(8161336) done Nov 05 01:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5640(8161842) WEXITSTATUS 1 Nov 05 01:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5640(8161842) done Nov 05 01:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157732 WEXITSTATUS 0 Nov 05 01:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157732 done Nov 05 01:34:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1287(8135277) WEXITSTATUS 0 Nov 05 01:34:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1287(8135277) done Nov 05 01:34:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_645(8161156) WEXITSTATUS 0 Nov 05 01:34:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_645(8161156) done Nov 05 01:34:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_749(8161344) WEXITSTATUS 0 Nov 05 01:34:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_749(8161344) done Nov 05 01:34:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5726(8162559) WEXITSTATUS 1 Nov 05 01:34:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5726(8162559) done Nov 05 01:34:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4761(8157332) WEXITSTATUS 1 Nov 05 01:34:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4761(8157332) done Nov 05 01:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_855(8161499) WEXITSTATUS 0 Nov 05 01:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_855(8161499) done Nov 05 01:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_729(8161324) WEXITSTATUS 0 Nov 05 01:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_729(8161324) done Nov 05 01:34:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162878 InitPrio=13461 usec=3788 Nov 05 01:34:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_600(8161111) WEXITSTATUS 0 Nov 05 01:34:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_600(8161111) done Nov 05 01:34:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_451(8160810) WEXITSTATUS 0 Nov 05 01:34:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_451(8160810) done Nov 05 01:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4373(8155200) WEXITSTATUS 1 Nov 05 01:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4373(8155200) done Nov 05 01:34:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_739(8161334) WEXITSTATUS 0 Nov 05 01:34:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_739(8161334) done Nov 05 01:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_714(8161309) WEXITSTATUS 0 Nov 05 01:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_714(8161309) done Nov 05 01:34:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2168(8126004) Nov 05 01:34:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8162879 InitPrio=13461 usec=3832 Nov 05 01:34:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131875 WEXITSTATUS 0 Nov 05 01:34:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131875 done Nov 05 01:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_605(8161116) WEXITSTATUS 0 Nov 05 01:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_605(8161116) done Nov 05 01:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_606(8161117) WEXITSTATUS 0 Nov 05 01:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_606(8161117) done Nov 05 01:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_738(8161333) WEXITSTATUS 0 Nov 05 01:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_738(8161333) done Nov 05 01:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_742(8161337) WEXITSTATUS 0 Nov 05 01:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_742(8161337) done Nov 05 01:34:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_746(8161341) WEXITSTATUS 0 Nov 05 01:34:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_746(8161341) done Nov 05 01:35:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162878_0(8162880) in pi_balou on r911u09n01 Nov 05 01:35:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162878_1(8162881) in pi_balou on r911u09n01 Nov 05 01:35:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162878_2(8162882) in pi_balou on r911u09n01 Nov 05 01:35:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162878_3(8162883) in pi_balou on r911u09n01 Nov 05 01:35:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162878_4(8162884) in pi_balou on r911u13n01 Nov 05 01:35:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162878_5(8162885) in pi_balou on r911u13n01 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162878_6(8162878) in pi_balou on r911u13n01 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162879_0(8162886) in pi_balou on r911u13n01 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162879_1(8162887) in pi_balou on r911u24n02 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162879_2(8162888) in pi_balou on r911u24n02 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162879_3(8162889) in pi_balou on r911u24n02 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162879_4(8162890) in pi_balou on r911u24n02 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162879_5(8162891) in pi_balou on r911u24n02 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162879_6(8162879) in pi_balou on r911u28n01 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_608(8161119) WEXITSTATUS 0 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_608(8161119) done Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3661(8162892) in gpu on r808u17n01 Nov 05 01:35:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154663 in day on r816u09n04 Nov 05 01:35:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1891(8162893) in day on r815u09n02 Nov 05 01:35:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133394 in gpu on r806u41n01 Nov 05 01:35:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 01:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_823(8161467) WEXITSTATUS 0 Nov 05 01:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_823(8161467) done Nov 05 01:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_824(8161468) WEXITSTATUS 0 Nov 05 01:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_824(8161468) done Nov 05 01:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_602(8161113) WEXITSTATUS 0 Nov 05 01:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_602(8161113) done Nov 05 01:35:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_265(8160473) WEXITSTATUS 0 Nov 05 01:35:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_265(8160473) done Nov 05 01:35:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_740(8161335) WEXITSTATUS 0 Nov 05 01:35:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_740(8161335) done Nov 05 01:35:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5902(8162894) in scavenge on r903u01n01 Nov 05 01:35:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5903(8162895) in scavenge on r915u35n04 Nov 05 01:35:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5904(8162896) in scavenge on r802u06n03 Nov 05 01:35:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5905(8162897) in scavenge on r802u14n03 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5906(8162898) in scavenge on r815u15n12 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5907(8162899) in scavenge on r816u15n10 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5908(8162900) in scavenge on r816u15n10 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5909(8162901) in scavenge on r815u23n09 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5910(8162902) in scavenge on r815u23n12 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5911(8162903) in scavenge on r816u23n08 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5912(8162904) in scavenge on r816u23n11 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5913(8162905) in scavenge on r808u14n01 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5914(8162906) in scavenge on r806u09n04 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5915(8162907) in scavenge on r919u36n01 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5916(8162908) in scavenge on r913u01n02 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5917(8162909) in scavenge on r913u24n04 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5918(8162910) in scavenge on r815u35n02 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5919(8162911) in scavenge on r815u35n02 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5920(8162912) in scavenge on r815u35n02 Nov 05 01:35:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5921(8162913) in scavenge on r816u03n06 Nov 05 01:35:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_455(8160814) WEXITSTATUS 0 Nov 05 01:35:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_455(8160814) done Nov 05 01:35:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_252(7572420) in scavenge on r816u15n10 Nov 05 01:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1461(8138138) in scavenge on r816u23n08 Nov 05 01:35:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 01:35:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_604(8161115) WEXITSTATUS 0 Nov 05 01:35:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_604(8161115) done Nov 05 01:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4607(8156183) WEXITSTATUS 1 Nov 05 01:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4607(8156183) done Nov 05 01:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157729 WEXITSTATUS 0 Nov 05 01:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157729 done Nov 05 01:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157725 WEXITSTATUS 0 Nov 05 01:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157725 done Nov 05 01:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157724 WEXITSTATUS 0 Nov 05 01:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157724 done Nov 05 01:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5066(8158371) WEXITSTATUS 1 Nov 05 01:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5066(8158371) done Nov 05 01:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5549(8161039) WEXITSTATUS 1 Nov 05 01:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5549(8161039) done Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:35:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:35:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4299(8154065) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:35:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_115(8160241) WEXITSTATUS 0 Nov 05 01:35:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_115(8160241) done Nov 05 01:35:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157715 WEXITSTATUS 0 Nov 05 01:35:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157715 done Nov 05 01:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_450(8160809) WEXITSTATUS 0 Nov 05 01:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_450(8160809) done Nov 05 01:35:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4299(8154065) Nov 05 01:35:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_827(8161471) WEXITSTATUS 0 Nov 05 01:35:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_827(8161471) done Nov 05 01:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157716 WEXITSTATUS 0 Nov 05 01:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157716 done Nov 05 01:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_614(8161125) WEXITSTATUS 0 Nov 05 01:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_614(8161125) done Nov 05 01:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_853(8161497) WEXITSTATUS 0 Nov 05 01:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_853(8161497) done Nov 05 01:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_611(8161122) WEXITSTATUS 0 Nov 05 01:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_611(8161122) done Nov 05 01:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_518(8160953) WEXITSTATUS 0 Nov 05 01:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_518(8160953) done Nov 05 01:35:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_601(8161112) WEXITSTATUS 0 Nov 05 01:35:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_601(8161112) done Nov 05 01:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_722(8161317) WEXITSTATUS 0 Nov 05 01:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_722(8161317) done Nov 05 01:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_642(8161153) WEXITSTATUS 0 Nov 05 01:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_642(8161153) done Nov 05 01:35:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1516(8143891) WEXITSTATUS 0 Nov 05 01:35:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1516(8143891) done Nov 05 01:35:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_646(8161157) WEXITSTATUS 0 Nov 05 01:35:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_646(8161157) done Nov 05 01:35:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_730(8161325) WEXITSTATUS 0 Nov 05 01:35:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_730(8161325) done Nov 05 01:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5498(8160335) WEXITSTATUS 1 Nov 05 01:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5498(8160335) done Nov 05 01:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3660(8162850) WEXITSTATUS 0 Nov 05 01:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3660(8162850) done Nov 05 01:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5496(8160333) WEXITSTATUS 1 Nov 05 01:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5496(8160333) done Nov 05 01:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_707(8161302) WEXITSTATUS 0 Nov 05 01:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_707(8161302) done Nov 05 01:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2234(8126134) WEXITSTATUS 1 Nov 05 01:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2234(8126134) done Nov 05 01:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154638 WEXITSTATUS 0 Nov 05 01:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154638 done Nov 05 01:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5917(8162909) WEXITSTATUS 1 Nov 05 01:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5917(8162909) done Nov 05 01:35:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_644(8161155) WEXITSTATUS 0 Nov 05 01:35:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_644(8161155) done Nov 05 01:36:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_312(8160605) WEXITSTATUS 0 Nov 05 01:36:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_312(8160605) done Nov 05 01:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_829(8161473) WEXITSTATUS 0 Nov 05 01:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_829(8161473) done Nov 05 01:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_832(8161476) WEXITSTATUS 0 Nov 05 01:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_832(8161476) done Nov 05 01:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5167(8158521) WEXITSTATUS 1 Nov 05 01:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5167(8158521) done Nov 05 01:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157731 WEXITSTATUS 0 Nov 05 01:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157731 done Nov 05 01:36:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_720(8161315) WEXITSTATUS 0 Nov 05 01:36:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_720(8161315) done Nov 05 01:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_706(8161301) WEXITSTATUS 0 Nov 05 01:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_706(8161301) done Nov 05 01:36:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_586(8161021) WEXITSTATUS 0 Nov 05 01:36:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_586(8161021) done Nov 05 01:36:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_616(8161127) WEXITSTATUS 0 Nov 05 01:36:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_616(8161127) done Nov 05 01:36:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3617(8161398) WEXITSTATUS 0 Nov 05 01:36:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3617(8161398) done Nov 05 01:36:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_597(8161032) WEXITSTATUS 0 Nov 05 01:36:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_597(8161032) done Nov 05 01:36:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_732(8161327) WEXITSTATUS 0 Nov 05 01:36:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_732(8161327) done Nov 05 01:36:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:36:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5437(8159561) WEXITSTATUS 1 Nov 05 01:36:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5437(8159561) done Nov 05 01:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_106(8160232) WEXITSTATUS 0 Nov 05 01:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_106(8160232) done Nov 05 01:36:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157733 WEXITSTATUS 0 Nov 05 01:36:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157733 done Nov 05 01:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1557(8147568) WEXITSTATUS 0 Nov 05 01:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1557(8147568) done Nov 05 01:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_826(8161470) WEXITSTATUS 0 Nov 05 01:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_826(8161470) done Nov 05 01:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_820(8161464) WEXITSTATUS 0 Nov 05 01:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_820(8161464) done Nov 05 01:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_831(8161475) WEXITSTATUS 0 Nov 05 01:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_831(8161475) done Nov 05 01:36:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_745(8161340) WEXITSTATUS 0 Nov 05 01:36:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_745(8161340) done Nov 05 01:36:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5068(8158373) WEXITSTATUS 1 Nov 05 01:36:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5068(8158373) done Nov 05 01:36:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_822(8161466) WEXITSTATUS 0 Nov 05 01:36:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_822(8161466) done Nov 05 01:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_727(8161322) WEXITSTATUS 0 Nov 05 01:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_727(8161322) done Nov 05 01:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_718(8161313) WEXITSTATUS 0 Nov 05 01:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_718(8161313) done Nov 05 01:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_725(8161320) WEXITSTATUS 0 Nov 05 01:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_725(8161320) done Nov 05 01:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_618(8161129) WEXITSTATUS 0 Nov 05 01:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_618(8161129) done Nov 05 01:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_728(8161323) WEXITSTATUS 0 Nov 05 01:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_728(8161323) done Nov 05 01:36:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_617(8161128) WEXITSTATUS 0 Nov 05 01:36:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_617(8161128) done Nov 05 01:36:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_615(8161126) WEXITSTATUS 0 Nov 05 01:36:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_615(8161126) done Nov 05 01:36:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_726(8161321) WEXITSTATUS 0 Nov 05 01:36:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_726(8161321) done Nov 05 01:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_638(8161149) WEXITSTATUS 0 Nov 05 01:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_638(8161149) done Nov 05 01:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_613(8161124) WEXITSTATUS 0 Nov 05 01:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_613(8161124) done Nov 05 01:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154613 WEXITSTATUS 0 Nov 05 01:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154613 done Nov 05 01:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162429 WEXITSTATUS 1 Nov 05 01:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162429 done Nov 05 01:36:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_629(8161140) WEXITSTATUS 0 Nov 05 01:36:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_629(8161140) done Nov 05 01:36:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_641(8161152) WEXITSTATUS 0 Nov 05 01:36:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_641(8161152) done Nov 05 01:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907587 WEXITSTATUS 0 Nov 05 01:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907587 done Nov 05 01:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_733(8161328) WEXITSTATUS 0 Nov 05 01:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_733(8161328) done Nov 05 01:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1538(8146141) WEXITSTATUS 0 Nov 05 01:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1538(8146141) done Nov 05 01:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5845(8162816) WEXITSTATUS 1 Nov 05 01:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5845(8162816) done Nov 05 01:37:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1309(8135367) WEXITSTATUS 0 Nov 05 01:37:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1309(8135367) done Nov 05 01:37:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_153(8160279) WEXITSTATUS 0 Nov 05 01:37:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_153(8160279) done Nov 05 01:37:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_271(8160479) WEXITSTATUS 0 Nov 05 01:37:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_271(8160479) done Nov 05 01:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_194(8160320) WEXITSTATUS 0 Nov 05 01:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_194(8160320) done Nov 05 01:37:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_825(8161469) WEXITSTATUS 0 Nov 05 01:37:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_825(8161469) done Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_833(8161477) WEXITSTATUS 0 Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_833(8161477) done Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_835(8161479) WEXITSTATUS 0 Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_835(8161479) done Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_830(8161474) WEXITSTATUS 0 Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_830(8161474) done Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_828(8161472) WEXITSTATUS 0 Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_828(8161472) done Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_719(8161314) WEXITSTATUS 0 Nov 05 01:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_719(8161314) done Nov 05 01:37:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_610(8161121) WEXITSTATUS 0 Nov 05 01:37:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_610(8161121) done Nov 05 01:37:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_627(8161138) WEXITSTATUS 0 Nov 05 01:37:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_627(8161138) done Nov 05 01:37:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_705(8161300) WEXITSTATUS 0 Nov 05 01:37:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_705(8161300) done Nov 05 01:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_731(8161326) WEXITSTATUS 0 Nov 05 01:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_731(8161326) done Nov 05 01:37:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_821(8161465) WEXITSTATUS 0 Nov 05 01:37:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_821(8161465) done Nov 05 01:37:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_609(8161120) WEXITSTATUS 0 Nov 05 01:37:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_609(8161120) done Nov 05 01:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_643(8161154) WEXITSTATUS 0 Nov 05 01:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_643(8161154) done Nov 05 01:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_612(8161123) WEXITSTATUS 0 Nov 05 01:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_612(8161123) done Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:37:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5872(8162844) WEXITSTATUS 1 Nov 05 01:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5872(8162844) done Nov 05 01:37:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_572(8161007) WEXITSTATUS 0 Nov 05 01:37:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_572(8161007) done Nov 05 01:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1305(8135358) WEXITSTATUS 0 Nov 05 01:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1305(8135358) done Nov 05 01:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159919 WEXITSTATUS 0 Nov 05 01:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159919 done Nov 05 01:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162660 in day on r805u14n04 Nov 05 01:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162832 in day on r918u01n03,r918u09n04 Nov 05 01:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907588 in gpu on r805u34n01 Nov 05 01:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3662(8162914) in gpu on r4513u26n01 Nov 05 01:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3663(8162915) in gpu on r903u34n01 Nov 05 01:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154661 in day on r806u11n02 Nov 05 01:37:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154667 in day on r815u35n03 Nov 05 01:37:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1892(8162916) in day on r816u15n05 Nov 05 01:37:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_878(8161522) WEXITSTATUS 0 Nov 05 01:37:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_878(8161522) done Nov 05 01:37:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_834(8161478) WEXITSTATUS 0 Nov 05 01:37:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_834(8161478) done Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5922(8162917) in scavenge on r816u15n12 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5923(8162918) in scavenge on r815u23n08 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5924(8162919) in scavenge on r815u23n12 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5925(8162920) in scavenge on r815u23n12 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5926(8162921) in scavenge on r815u23n12 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5927(8162922) in scavenge on r816u23n11 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5928(8162923) in scavenge on r816u29n09 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5929(8162924) in scavenge on r816u29n10 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5930(8162925) in scavenge on r903u40n01 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5931(8162926) in scavenge on r806u02n03 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5932(8162927) in scavenge on r904u07n04 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5933(8162928) in scavenge on r904u10n03 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5934(8162929) in scavenge on r805u02n04 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5935(8162930) in scavenge on r805u11n01 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5936(8162931) in scavenge on r912u28n04 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5937(8162932) in scavenge on r912u36n03 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5938(8162933) in scavenge on r913u09n02 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5939(8162934) in scavenge on r913u24n04 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5940(8162935) in scavenge on r815u09n08 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5941(8162936) in scavenge on r815u35n02 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5942(8162937) in scavenge on r816u15n01 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5943(8162938) in scavenge on r816u15n01 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5944(8162939) in scavenge on r816u35n06 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5945(8162940) in scavenge on r805u38n01 Nov 05 01:37:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5946(8162941) in scavenge on r806u41n01 Nov 05 01:37:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_724(8161319) WEXITSTATUS 0 Nov 05 01:37:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_724(8161319) done Nov 05 01:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_723(8161318) WEXITSTATUS 0 Nov 05 01:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_723(8161318) done Nov 05 01:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_34(8158666) WEXITSTATUS 0 Nov 05 01:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_34(8158666) done Nov 05 01:38:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158123 Nov 05 01:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_703(8161298) WEXITSTATUS 0 Nov 05 01:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_703(8161298) done Nov 05 01:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_640(8161151) WEXITSTATUS 0 Nov 05 01:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_640(8161151) done Nov 05 01:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5834(8162801) WEXITSTATUS 1 Nov 05 01:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5834(8162801) done Nov 05 01:38:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157735 WEXITSTATUS 0 Nov 05 01:38:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157735 done Nov 05 01:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_920(8161589) WEXITSTATUS 0 Nov 05 01:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_920(8161589) done Nov 05 01:38:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_879(8161523) WEXITSTATUS 0 Nov 05 01:38:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_879(8161523) done Nov 05 01:38:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_704(8161299) WEXITSTATUS 0 Nov 05 01:38:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_704(8161299) done Nov 05 01:38:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_700(8161295) WEXITSTATUS 0 Nov 05 01:38:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_700(8161295) done Nov 05 01:38:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_607(8161118) WEXITSTATUS 0 Nov 05 01:38:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_607(8161118) done Nov 05 01:38:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2900(8128584) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:38:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4715(8156726) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:38:34 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4900(8158033) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:38:34 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8162189 had to be killed Nov 05 01:38:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1544(8146376) WEXITSTATUS 0 Nov 05 01:38:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1544(8146376) done Nov 05 01:38:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2900(8128584) Nov 05 01:38:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4715(8156726) Nov 05 01:38:37 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4900(8158033) Nov 05 01:38:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_17(8160040) WEXITSTATUS 0 Nov 05 01:38:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_17(8160040) done Nov 05 01:38:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_183(8160309) WEXITSTATUS 0 Nov 05 01:38:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_183(8160309) done Nov 05 01:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5598(8161674) WEXITSTATUS 1 Nov 05 01:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5598(8161674) done Nov 05 01:38:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5333(8158952) WEXITSTATUS 1 Nov 05 01:38:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5333(8158952) done Nov 05 01:39:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5942(8162937) WEXITSTATUS 1 Nov 05 01:39:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5942(8162937) done Nov 05 01:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_918(8161587) WEXITSTATUS 0 Nov 05 01:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_918(8161587) done Nov 05 01:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_701(8161296) WEXITSTATUS 0 Nov 05 01:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_701(8161296) done Nov 05 01:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_818(8161462) WEXITSTATUS 0 Nov 05 01:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_818(8161462) done Nov 05 01:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5850(8162821) WEXITSTATUS 1 Nov 05 01:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5850(8162821) done Nov 05 01:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159904 WEXITSTATUS 0 Nov 05 01:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159904 done Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:39:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:39:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_708(8161303) WEXITSTATUS 0 Nov 05 01:39:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_708(8161303) done Nov 05 01:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_210(8160418) WEXITSTATUS 0 Nov 05 01:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_210(8160418) done Nov 05 01:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3649(8162748) WEXITSTATUS 0 Nov 05 01:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3649(8162748) done Nov 05 01:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5685(8162307) WEXITSTATUS 1 Nov 05 01:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5685(8162307) done Nov 05 01:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_919(8161588) WEXITSTATUS 0 Nov 05 01:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_919(8161588) done Nov 05 01:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_917(8161586) WEXITSTATUS 0 Nov 05 01:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_917(8161586) done Nov 05 01:40:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5711(8162424) WEXITSTATUS 1 Nov 05 01:40:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5711(8162424) done Nov 05 01:40:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_875(8161519) WEXITSTATUS 0 Nov 05 01:40:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_875(8161519) done Nov 05 01:40:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_874(8161518) WEXITSTATUS 0 Nov 05 01:40:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_874(8161518) done Nov 05 01:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5602(8161689) WEXITSTATUS 1 Nov 05 01:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5602(8161689) done Nov 05 01:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131871 WEXITSTATUS 0 Nov 05 01:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8131871 done Nov 05 01:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_852(8161496) WEXITSTATUS 0 Nov 05 01:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_852(8161496) done Nov 05 01:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_819(8161463) WEXITSTATUS 0 Nov 05 01:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_819(8161463) done Nov 05 01:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_872(8161516) WEXITSTATUS 0 Nov 05 01:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_872(8161516) done Nov 05 01:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5739(8162595) WEXITSTATUS 1 Nov 05 01:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5739(8162595) done Nov 05 01:40:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_702(8161297) WEXITSTATUS 0 Nov 05 01:40:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_702(8161297) done Nov 05 01:40:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5768(8162704) WEXITSTATUS 1 Nov 05 01:40:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5768(8162704) done Nov 05 01:40:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162665 in day on r909u07n01 Nov 05 01:40:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3664(8162942) in gpu on r808u14n01 Nov 05 01:40:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154664 in day on r912u28n01 Nov 05 01:40:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154669 in day on r815u29n01 Nov 05 01:40:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154670 in day on r815u29n05 Nov 05 01:40:36 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_11(8157338) Nov 05 01:40:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133395 in gpu on r806u37n01 Nov 05 01:40:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 01:40:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1348(8135493) WEXITSTATUS 0 Nov 05 01:40:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1348(8135493) done Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5947(8162943) in scavenge on r802u30n03 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5948(8162944) in scavenge on r904u01n02 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5949(8162945) in scavenge on r816u15n11 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5950(8162946) in scavenge on r816u23n12 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5951(8162947) in scavenge on r808u11n01 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5952(8162948) in scavenge on r4513u23n01 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5953(8162949) in scavenge on r806u02n03 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5954(8162950) in scavenge on r909u07n01 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5955(8162951) in scavenge on r912u28n04 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5956(8162952) in scavenge on r816u35n04 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5957(8162953) in scavenge on r816u03n07 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5958(8162954) in scavenge on r816u03n07 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5959(8162955) in scavenge on r816u15n01 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5870(8162842) WEXITSTATUS 1 Nov 05 01:40:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5870(8162842) done Nov 05 01:40:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_264(7572432) in scavenge on r816u03n07 Nov 05 01:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1490(8139331) WEXITSTATUS 0 Nov 05 01:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1490(8139331) done Nov 05 01:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1447(8136422) WEXITSTATUS 0 Nov 05 01:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1447(8136422) done Nov 05 01:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_876(8161520) WEXITSTATUS 0 Nov 05 01:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_876(8161520) done Nov 05 01:40:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_898(8161542) WEXITSTATUS 0 Nov 05 01:40:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_898(8161542) done Nov 05 01:40:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3661(8162892) WEXITSTATUS 0 Nov 05 01:40:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3661(8162892) done Nov 05 01:40:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1545(8146377) WEXITSTATUS 0 Nov 05 01:40:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1545(8146377) done Nov 05 01:40:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154637 WEXITSTATUS 0 Nov 05 01:40:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154637 done Nov 05 01:41:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_973(8161642) WEXITSTATUS 0 Nov 05 01:41:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_973(8161642) done Nov 05 01:41:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_880(8161524) WEXITSTATUS 0 Nov 05 01:41:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_880(8161524) done Nov 05 01:41:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1352(8135520) WEXITSTATUS 0 Nov 05 01:41:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1352(8135520) done Nov 05 01:41:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_886(8161530) WEXITSTATUS 0 Nov 05 01:41:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_886(8161530) done Nov 05 01:41:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162614 WEXITSTATUS 0 Nov 05 01:41:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162614 done Nov 05 01:41:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_899(8161543) WEXITSTATUS 0 Nov 05 01:41:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_899(8161543) done Nov 05 01:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_850(8161494) WEXITSTATUS 0 Nov 05 01:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_850(8161494) done Nov 05 01:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_300(8160593) WEXITSTATUS 0 Nov 05 01:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_300(8160593) done Nov 05 01:41:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_165(8160291) WEXITSTATUS 0 Nov 05 01:41:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_165(8160291) done Nov 05 01:41:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5957(8162953) WEXITSTATUS 1 Nov 05 01:41:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5957(8162953) done Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:41:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:41:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5947(8162943) WEXITSTATUS 1 Nov 05 01:41:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5947(8162943) done Nov 05 01:41:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_873(8161517) WEXITSTATUS 0 Nov 05 01:41:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_873(8161517) done Nov 05 01:41:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_974(8161643) WEXITSTATUS 0 Nov 05 01:41:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_974(8161643) done Nov 05 01:41:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4949(8158194) WEXITSTATUS 1 Nov 05 01:41:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4949(8158194) done Nov 05 01:41:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_882(8161526) WEXITSTATUS 0 Nov 05 01:41:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_882(8161526) done Nov 05 01:41:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_896(8161540) WEXITSTATUS 0 Nov 05 01:41:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_896(8161540) done Nov 05 01:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_847(8161491) WEXITSTATUS 0 Nov 05 01:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_847(8161491) done Nov 05 01:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_851(8161495) WEXITSTATUS 0 Nov 05 01:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_851(8161495) done Nov 05 01:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_843(8161487) WEXITSTATUS 0 Nov 05 01:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_843(8161487) done Nov 05 01:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_842(8161486) WEXITSTATUS 0 Nov 05 01:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_842(8161486) done Nov 05 01:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_841(8161485) WEXITSTATUS 0 Nov 05 01:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_841(8161485) done Nov 05 01:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_849(8161493) WEXITSTATUS 0 Nov 05 01:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_849(8161493) done Nov 05 01:42:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_848(8161492) WEXITSTATUS 0 Nov 05 01:42:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_848(8161492) done Nov 05 01:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5542(8160877) WEXITSTATUS 1 Nov 05 01:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5542(8160877) done Nov 05 01:42:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2146(8125966) Nov 05 01:42:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2179(8126034) Nov 05 01:42:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157736 WEXITSTATUS 0 Nov 05 01:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157736 done Nov 05 01:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_953(8161622) WEXITSTATUS 0 Nov 05 01:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_953(8161622) done Nov 05 01:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5618(8161719) WEXITSTATUS 1 Nov 05 01:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5618(8161719) done Nov 05 01:42:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 01:42:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 01:42:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 01:42:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_888(8161532) WEXITSTATUS 0 Nov 05 01:42:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_888(8161532) done Nov 05 01:42:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_891(8161535) WEXITSTATUS 0 Nov 05 01:42:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_891(8161535) done Nov 05 01:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_892(8161536) WEXITSTATUS 0 Nov 05 01:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_892(8161536) done Nov 05 01:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_883(8161527) WEXITSTATUS 0 Nov 05 01:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_883(8161527) done Nov 05 01:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_335(8160628) WEXITSTATUS 0 Nov 05 01:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_335(8160628) done Nov 05 01:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_916(8161585) WEXITSTATUS 0 Nov 05 01:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_916(8161585) done Nov 05 01:42:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_893(8161537) WEXITSTATUS 0 Nov 05 01:42:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_893(8161537) done Nov 05 01:42:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_839(8161483) WEXITSTATUS 0 Nov 05 01:42:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_839(8161483) done Nov 05 01:42:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_871(8161515) WEXITSTATUS 0 Nov 05 01:42:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_871(8161515) done Nov 05 01:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_838(8161482) WEXITSTATUS 0 Nov 05 01:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_838(8161482) done Nov 05 01:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5753(8162618) WEXITSTATUS 1 Nov 05 01:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5753(8162618) done Nov 05 01:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_24(8158624) WEXITSTATUS 0 Nov 05 01:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_24(8158624) done Nov 05 01:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_639(8161150) WEXITSTATUS 0 Nov 05 01:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_639(8161150) done Nov 05 01:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5883(8162859) WEXITSTATUS 1 Nov 05 01:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5883(8162859) done Nov 05 01:42:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5456(8159772) WEXITSTATUS 1 Nov 05 01:42:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5456(8159772) done Nov 05 01:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_244(8160452) WEXITSTATUS 0 Nov 05 01:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_244(8160452) done Nov 05 01:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1284(8135267) WEXITSTATUS 0 Nov 05 01:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1284(8135267) done Nov 05 01:42:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5893(8162869) WEXITSTATUS 1 Nov 05 01:42:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5893(8162869) done Nov 05 01:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5878(8162854) WEXITSTATUS 1 Nov 05 01:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5878(8162854) done Nov 05 01:42:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_877(8161521) WEXITSTATUS 0 Nov 05 01:42:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_877(8161521) done Nov 05 01:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_897(8161541) WEXITSTATUS 0 Nov 05 01:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_897(8161541) done Nov 05 01:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_836(8161480) WEXITSTATUS 0 Nov 05 01:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_836(8161480) done Nov 05 01:43:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5826(8162788) WEXITSTATUS 1 Nov 05 01:43:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5826(8162788) done Nov 05 01:43:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3665(8162956) in gpu on r808u17n01 Nov 05 01:43:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1893(8162957) in day on r815u09n05 Nov 05 01:43:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1894(8162958) in day on r815u23n04 Nov 05 01:43:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1895(8162959) in day on r816u15n04 Nov 05 01:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1497(8141692) WEXITSTATUS 0 Nov 05 01:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1497(8141692) done Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5960(8162960) in scavenge on r908u35n02 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5961(8162961) in scavenge on r908u35n02 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5962(8162962) in scavenge on r801u04n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5963(8162963) in scavenge on r903u04n03 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5964(8162964) in scavenge on r802u04n04 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5965(8162965) in scavenge on r802u30n03 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5966(8162966) in scavenge on r815u23n07 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5967(8162967) in scavenge on r816u23n08 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5968(8162968) in scavenge on r903u40n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5969(8162969) in scavenge on r904u04n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5970(8162970) in scavenge on r805u02n02 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5971(8162971) in scavenge on r805u09n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5972(8162972) in scavenge on r912u28n02 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5973(8162973) in scavenge on r912u28n02 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5974(8162974) in scavenge on r913u05n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5975(8162975) in scavenge on r918u01n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5976(8162976) in scavenge on r918u05n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5977(8162977) in scavenge on r918u05n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5978(8162978) in scavenge on r918u05n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5979(8162979) in scavenge on r918u05n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5980(8162980) in scavenge on r918u05n02 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5981(8162981) in scavenge on r918u05n02 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5982(8162982) in scavenge on r918u36n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5983(8162983) in scavenge on r918u36n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5984(8162984) in scavenge on r918u36n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5985(8162985) in scavenge on r918u36n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5986(8162986) in scavenge on r919u01n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5987(8162987) in scavenge on r919u01n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5988(8162988) in scavenge on r919u01n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5989(8162989) in scavenge on r919u05n03 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5990(8162990) in scavenge on r919u05n04 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5991(8162991) in scavenge on r919u05n04 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5992(8162992) in scavenge on r919u09n01 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5993(8162993) in scavenge on r816u03n07 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_964(8161633) WEXITSTATUS 0 Nov 05 01:43:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_964(8161633) done Nov 05 01:43:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_284(7572452) in scavenge on r816u15n11 Nov 05 01:43:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_399(8160692) WEXITSTATUS 0 Nov 05 01:43:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_399(8160692) done Nov 05 01:43:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154630 WEXITSTATUS 0 Nov 05 01:43:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154630 done Nov 05 01:43:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_975(8161644) WEXITSTATUS 0 Nov 05 01:43:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_975(8161644) done Nov 05 01:43:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5861(8162833) WEXITSTATUS 1 Nov 05 01:43:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5861(8162833) done Nov 05 01:43:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_914(8161583) WEXITSTATUS 0 Nov 05 01:43:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_914(8161583) done Nov 05 01:43:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_954(8161623) WEXITSTATUS 0 Nov 05 01:43:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_954(8161623) done Nov 05 01:43:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_894(8161538) WEXITSTATUS 0 Nov 05 01:43:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_894(8161538) done Nov 05 01:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_890(8161534) WEXITSTATUS 0 Nov 05 01:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_890(8161534) done Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_885(8161529) WEXITSTATUS 0 Nov 05 01:43:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_885(8161529) done Nov 05 01:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_837(8161481) WEXITSTATUS 0 Nov 05 01:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_837(8161481) done Nov 05 01:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5065(8158370) WEXITSTATUS 1 Nov 05 01:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5065(8158370) done Nov 05 01:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_840(8161484) WEXITSTATUS 0 Nov 05 01:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_840(8161484) done Nov 05 01:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_895(8161539) WEXITSTATUS 0 Nov 05 01:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_895(8161539) done Nov 05 01:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_881(8161525) WEXITSTATUS 0 Nov 05 01:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_881(8161525) done Nov 05 01:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_915(8161584) WEXITSTATUS 0 Nov 05 01:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_915(8161584) done Nov 05 01:43:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1518(8143998) WEXITSTATUS 0 Nov 05 01:43:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1518(8143998) done Nov 05 01:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5880(8162856) WEXITSTATUS 1 Nov 05 01:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5880(8162856) done Nov 05 01:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154625 WEXITSTATUS 0 Nov 05 01:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154625 done Nov 05 01:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154645 WEXITSTATUS 0 Nov 05 01:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154645 done Nov 05 01:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5898(8162874) WEXITSTATUS 1 Nov 05 01:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5898(8162874) done Nov 05 01:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162569 WEXITSTATUS 0 Nov 05 01:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162569 done Nov 05 01:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162529 WEXITSTATUS 0 Nov 05 01:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162529 done Nov 05 01:44:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157734 WEXITSTATUS 0 Nov 05 01:44:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157734 done Nov 05 01:44:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133210 WEXITSTATUS 0 Nov 05 01:44:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133210 done Nov 05 01:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_959(8161628) WEXITSTATUS 0 Nov 05 01:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_959(8161628) done Nov 05 01:44:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3507(8135534) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:44:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_12(8157464) Nov 05 01:44:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5697(8162359) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:44:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5715(8162480) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:44:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5981(8162981) WEXITSTATUS 1 Nov 05 01:44:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5981(8162981) done Nov 05 01:44:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3507(8135534) Nov 05 01:44:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5715(8162480) Nov 05 01:44:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5697(8162359) Nov 05 01:44:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5017(8158308) WEXITSTATUS 1 Nov 05 01:44:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5017(8158308) done Nov 05 01:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_887(8161531) WEXITSTATUS 0 Nov 05 01:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_887(8161531) done Nov 05 01:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1622(8150601) WEXITSTATUS 0 Nov 05 01:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_1622(8150601) done Nov 05 01:44:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_845(8161489) WEXITSTATUS 0 Nov 05 01:44:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_845(8161489) done Nov 05 01:44:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_846(8161490) WEXITSTATUS 0 Nov 05 01:44:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_846(8161490) done Nov 05 01:44:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5906(8162898) WEXITSTATUS 1 Nov 05 01:44:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5906(8162898) done Nov 05 01:44:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_650(8162598) WEXITSTATUS 0 Nov 05 01:44:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_650(8162598) done Nov 05 01:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_956(8161625) WEXITSTATUS 0 Nov 05 01:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_956(8161625) done Nov 05 01:44:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_967(8161636) WEXITSTATUS 0 Nov 05 01:44:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_967(8161636) done Nov 05 01:44:37 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8160345 Nov 05 01:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5779(8162729) WEXITSTATUS 1 Nov 05 01:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5779(8162729) done Nov 05 01:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_966(8161635) WEXITSTATUS 0 Nov 05 01:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_966(8161635) done Nov 05 01:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_968(8161637) WEXITSTATUS 0 Nov 05 01:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_968(8161637) done Nov 05 01:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_957(8161626) WEXITSTATUS 0 Nov 05 01:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_957(8161626) done Nov 05 01:44:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_435(8160794) WEXITSTATUS 0 Nov 05 01:44:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_435(8160794) done Nov 05 01:44:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_955(8161624) WEXITSTATUS 0 Nov 05 01:44:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_955(8161624) done Nov 05 01:44:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_889(8161533) WEXITSTATUS 0 Nov 05 01:44:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_889(8161533) done Nov 05 01:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_815(8161459) WEXITSTATUS 0 Nov 05 01:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_815(8161459) done Nov 05 01:44:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_884(8161528) WEXITSTATUS 0 Nov 05 01:44:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_884(8161528) done Nov 05 01:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162747 WEXITSTATUS 0 Nov 05 01:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162747 done Nov 05 01:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_806(8161450) WEXITSTATUS 0 Nov 05 01:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_806(8161450) done Nov 05 01:44:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2380(8152424) WEXITSTATUS 0 Nov 05 01:44:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2380(8152424) done Nov 05 01:44:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3662(8162914) WEXITSTATUS 0 Nov 05 01:44:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3662(8162914) done Nov 05 01:44:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3665(8162956) WEXITSTATUS 0 Nov 05 01:44:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3665(8162956) done Nov 05 01:44:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162568 WEXITSTATUS 0 Nov 05 01:44:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162568 done Nov 05 01:45:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_813(8161457) WEXITSTATUS 0 Nov 05 01:45:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_813(8161457) done Nov 05 01:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4262(8153357) WEXITSTATUS 1 Nov 05 01:45:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4262(8153357) done Nov 05 01:45:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2184(8126043) Nov 05 01:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_933(8161602) WEXITSTATUS 0 Nov 05 01:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_933(8161602) done Nov 05 01:45:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_338(8160631) WEXITSTATUS 0 Nov 05 01:45:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_338(8160631) done Nov 05 01:45:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_963(8161632) WEXITSTATUS 0 Nov 05 01:45:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_963(8161632) done Nov 05 01:45:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_844(8161488) WEXITSTATUS 0 Nov 05 01:45:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_844(8161488) done Nov 05 01:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1489(8139330) WEXITSTATUS 0 Nov 05 01:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1489(8139330) done Nov 05 01:45:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:45:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:45:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:45:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:45:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 01:45:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3664(8162942) WEXITSTATUS 0 Nov 05 01:45:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3664(8162942) done Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:45:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:45:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5492(8160329) WEXITSTATUS 1 Nov 05 01:45:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5492(8160329) done Nov 05 01:45:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5840(8162807) WEXITSTATUS 1 Nov 05 01:45:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5840(8162807) done Nov 05 01:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_960(8161629) WEXITSTATUS 0 Nov 05 01:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_960(8161629) done Nov 05 01:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2176(8151968) WEXITSTATUS 0 Nov 05 01:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2176(8151968) done Nov 05 01:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5772(8162714) WEXITSTATUS 1 Nov 05 01:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5772(8162714) done Nov 05 01:45:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_961(8161630) WEXITSTATUS 0 Nov 05 01:45:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_961(8161630) done Nov 05 01:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_970(8161639) WEXITSTATUS 0 Nov 05 01:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_970(8161639) done Nov 05 01:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_962(8161631) WEXITSTATUS 0 Nov 05 01:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_962(8161631) done Nov 05 01:45:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_801(8161445) WEXITSTATUS 0 Nov 05 01:45:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_801(8161445) done Nov 05 01:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_811(8161455) WEXITSTATUS 0 Nov 05 01:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_811(8161455) done Nov 05 01:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_4(8162994) in pi_ohern on r917u29n01 Nov 05 01:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162666 in day on r805u06n03 Nov 05 01:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162667 in day on r805u11n01 Nov 05 01:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162668 in day on r918u05n02 Nov 05 01:46:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3666(8162995) in gpu on r808u14n01 Nov 05 01:46:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3667(8162996) in gpu on r808u17n01 Nov 05 01:46:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3668(8162997) in gpu on r4513u26n01 Nov 05 01:46:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154666 in day on r919u36n03 Nov 05 01:46:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154672 in day on r815u35n03 Nov 05 01:46:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154673 in day on r816u03n04 Nov 05 01:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_663(8114932) WEXITSTATUS 0 Nov 05 01:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_663(8114932) done Nov 05 01:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_0(8162880) WEXITSTATUS 0 Nov 05 01:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_0(8162880) done Nov 05 01:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_803(8161447) WEXITSTATUS 0 Nov 05 01:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_803(8161447) done Nov 05 01:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1896(8162998) in day on r816u15n06 Nov 05 01:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5994(8162999) in scavenge on r907u05n04 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5995(8163000) in scavenge on r801u23n01 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5996(8163001) in scavenge on r802u32n04 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5997(8163002) in scavenge on r911u24n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5998(8163003) in scavenge on r816u29n08 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5999(8163004) in scavenge on r815u09n12 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6000(8163005) in scavenge on r815u15n12 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6001(8163006) in scavenge on r805u09n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6002(8163007) in scavenge on r805u09n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6003(8163008) in scavenge on r805u09n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6004(8163009) in scavenge on r909u07n03 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6005(8163010) in scavenge on r918u09n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6006(8163011) in scavenge on r918u09n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6007(8163012) in scavenge on r918u09n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6008(8163013) in scavenge on r918u09n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6009(8163014) in scavenge on r918u36n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6010(8163015) in scavenge on r918u36n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6011(8163016) in scavenge on r918u36n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6012(8163017) in scavenge on r918u36n02 Nov 05 01:46:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6013(8163018) in scavenge on r815u35n03 Nov 05 01:46:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_497(8160856) WEXITSTATUS 0 Nov 05 01:46:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_497(8160856) done Nov 05 01:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_303(7572471) in scavenge on r805u11n01 Nov 05 01:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1373(8135668) WEXITSTATUS 0 Nov 05 01:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1373(8135668) done Nov 05 01:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_691(8161202) WEXITSTATUS 0 Nov 05 01:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_691(8161202) done Nov 05 01:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1304(8135328) WEXITSTATUS 0 Nov 05 01:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1304(8135328) done Nov 05 01:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1511(8143570) in scavenge on r816u29n08 Nov 05 01:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_0(8162886) WEXITSTATUS 0 Nov 05 01:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_0(8162886) done Nov 05 01:46:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_958(8161627) WEXITSTATUS 0 Nov 05 01:46:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_958(8161627) done Nov 05 01:46:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_948(8161617) WEXITSTATUS 0 Nov 05 01:46:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_948(8161617) done Nov 05 01:46:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157757 WEXITSTATUS 0 Nov 05 01:46:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157757 done Nov 05 01:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5205(8158568) WEXITSTATUS 1 Nov 05 01:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5205(8158568) done Nov 05 01:46:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_807(8161451) WEXITSTATUS 0 Nov 05 01:46:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_807(8161451) done Nov 05 01:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6010(8163015) WEXITSTATUS 1 Nov 05 01:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6010(8163015) done Nov 05 01:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_802(8161446) WEXITSTATUS 0 Nov 05 01:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_802(8161446) done Nov 05 01:46:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3658(8162811) WEXITSTATUS 0 Nov 05 01:46:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3658(8162811) done Nov 05 01:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_947(8161616) WEXITSTATUS 0 Nov 05 01:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_947(8161616) done Nov 05 01:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_929(8161598) WEXITSTATUS 0 Nov 05 01:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_929(8161598) done Nov 05 01:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_908(8161577) WEXITSTATUS 0 Nov 05 01:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_908(8161577) done Nov 05 01:46:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_939(8161608) WEXITSTATUS 0 Nov 05 01:46:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_939(8161608) done Nov 05 01:46:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_868(8161512) WEXITSTATUS 0 Nov 05 01:46:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_868(8161512) done Nov 05 01:46:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_859(8161503) WEXITSTATUS 0 Nov 05 01:46:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_859(8161503) done Nov 05 01:46:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_865(8161509) WEXITSTATUS 0 Nov 05 01:46:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_865(8161509) done Nov 05 01:46:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_969(8161638) WEXITSTATUS 0 Nov 05 01:46:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_969(8161638) done Nov 05 01:46:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_519(8160954) WEXITSTATUS 0 Nov 05 01:46:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_519(8160954) done Nov 05 01:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5799(8162758) WEXITSTATUS 1 Nov 05 01:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5799(8162758) done Nov 05 01:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_804(8161448) WEXITSTATUS 0 Nov 05 01:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_804(8161448) done Nov 05 01:47:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_800(8161444) WEXITSTATUS 0 Nov 05 01:47:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_800(8161444) done Nov 05 01:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_816(8161460) WEXITSTATUS 0 Nov 05 01:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_816(8161460) done Nov 05 01:47:12 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3490(8157951) WEXITSTATUS 0 Nov 05 01:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3490(8157951) done Nov 05 01:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_812(8161456) WEXITSTATUS 0 Nov 05 01:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_812(8161456) done Nov 05 01:47:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154612 WEXITSTATUS 0 Nov 05 01:47:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154612 done Nov 05 01:47:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_943(8161612) WEXITSTATUS 0 Nov 05 01:47:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_943(8161612) done Nov 05 01:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_573(8161008) WEXITSTATUS 0 Nov 05 01:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_573(8161008) done Nov 05 01:47:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_814(8161458) WEXITSTATUS 0 Nov 05 01:47:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_814(8161458) done Nov 05 01:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_936(8161605) WEXITSTATUS 0 Nov 05 01:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_936(8161605) done Nov 05 01:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_950(8161619) WEXITSTATUS 0 Nov 05 01:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_950(8161619) done Nov 05 01:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_942(8161611) WEXITSTATUS 0 Nov 05 01:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_942(8161611) done Nov 05 01:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_937(8161606) WEXITSTATUS 0 Nov 05 01:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_937(8161606) done Nov 05 01:47:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1255(8135157) WEXITSTATUS 0 Nov 05 01:47:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1255(8135157) done Nov 05 01:47:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_938(8161607) WEXITSTATUS 0 Nov 05 01:47:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_938(8161607) done Nov 05 01:47:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_935(8161604) WEXITSTATUS 0 Nov 05 01:47:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_935(8161604) done Nov 05 01:47:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5944(8162939) WEXITSTATUS 1 Nov 05 01:47:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5944(8162939) done Nov 05 01:47:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_869(8161513) WEXITSTATUS 0 Nov 05 01:47:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_869(8161513) done Nov 05 01:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_863(8161507) WEXITSTATUS 0 Nov 05 01:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_863(8161507) done Nov 05 01:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_622(8161133) WEXITSTATUS 0 Nov 05 01:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_622(8161133) done Nov 05 01:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_926(8161595) WEXITSTATUS 0 Nov 05 01:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_926(8161595) done Nov 05 01:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_857(8161501) WEXITSTATUS 0 Nov 05 01:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_857(8161501) done Nov 05 01:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5580(8161400) WEXITSTATUS 1 Nov 05 01:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5580(8161400) done Nov 05 01:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5761(8162631) WEXITSTATUS 1 Nov 05 01:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5761(8162631) done Nov 05 01:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_932(8161601) WEXITSTATUS 0 Nov 05 01:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_932(8161601) done Nov 05 01:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_860(8161504) WEXITSTATUS 0 Nov 05 01:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_860(8161504) done Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_971(8161640) WEXITSTATUS 0 Nov 05 01:47:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_971(8161640) done Nov 05 01:47:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_809(8161453) WEXITSTATUS 0 Nov 05 01:47:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_809(8161453) done Nov 05 01:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5597(8161673) WEXITSTATUS 1 Nov 05 01:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5597(8161673) done Nov 05 01:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_808(8161452) WEXITSTATUS 0 Nov 05 01:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_808(8161452) done Nov 05 01:47:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5313(8158841) WEXITSTATUS 1 Nov 05 01:47:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5313(8158841) done Nov 05 01:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_900(8161569) WEXITSTATUS 0 Nov 05 01:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_900(8161569) done Nov 05 01:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_571(8161006) WEXITSTATUS 0 Nov 05 01:47:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_571(8161006) done Nov 05 01:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_810(8161454) WEXITSTATUS 0 Nov 05 01:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_810(8161454) done Nov 05 01:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_965(8161634) WEXITSTATUS 0 Nov 05 01:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_965(8161634) done Nov 05 01:47:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_925(8161594) WEXITSTATUS 0 Nov 05 01:47:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_925(8161594) done Nov 05 01:47:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_922(8161591) WEXITSTATUS 0 Nov 05 01:47:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_922(8161591) done Nov 05 01:47:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_924(8161593) WEXITSTATUS 0 Nov 05 01:47:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_924(8161593) done Nov 05 01:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_866(8161510) WEXITSTATUS 0 Nov 05 01:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_866(8161510) done Nov 05 01:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_928(8161597) WEXITSTATUS 0 Nov 05 01:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_928(8161597) done Nov 05 01:48:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_934(8161603) WEXITSTATUS 0 Nov 05 01:48:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_934(8161603) done Nov 05 01:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_930(8161599) WEXITSTATUS 0 Nov 05 01:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_930(8161599) done Nov 05 01:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3668(8162997) WEXITSTATUS 0 Nov 05 01:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3668(8162997) done Nov 05 01:48:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_951(8161620) WEXITSTATUS 0 Nov 05 01:48:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_951(8161620) done Nov 05 01:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_856(8161500) WEXITSTATUS 0 Nov 05 01:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_856(8161500) done Nov 05 01:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5747(8162608) WEXITSTATUS 1 Nov 05 01:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5747(8162608) done Nov 05 01:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_805(8161449) WEXITSTATUS 0 Nov 05 01:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_805(8161449) done Nov 05 01:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5950(8162946) WEXITSTATUS 1 Nov 05 01:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5950(8162946) done Nov 05 01:48:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_946(8161615) WEXITSTATUS 0 Nov 05 01:48:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_946(8161615) done Nov 05 01:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5689(8162311) WEXITSTATUS 1 Nov 05 01:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5689(8162311) done Nov 05 01:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1496(8141691) WEXITSTATUS 0 Nov 05 01:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1496(8141691) done Nov 05 01:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_945(8161614) WEXITSTATUS 0 Nov 05 01:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_945(8161614) done Nov 05 01:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_940(8161609) WEXITSTATUS 0 Nov 05 01:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_940(8161609) done Nov 05 01:48:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5939(8162934) WEXITSTATUS 1 Nov 05 01:48:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5939(8162934) done Nov 05 01:48:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5759(8162629) WEXITSTATUS 1 Nov 05 01:48:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5759(8162629) done Nov 05 01:48:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_901(8161570) WEXITSTATUS 0 Nov 05 01:48:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_901(8161570) done Nov 05 01:48:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_870(8161514) WEXITSTATUS 0 Nov 05 01:48:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_870(8161514) done Nov 05 01:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_923(8161592) WEXITSTATUS 0 Nov 05 01:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_923(8161592) done Nov 05 01:48:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_862(8161506) WEXITSTATUS 0 Nov 05 01:48:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_862(8161506) done Nov 05 01:48:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_931(8161600) WEXITSTATUS 0 Nov 05 01:48:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_931(8161600) done Nov 05 01:48:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_949(8161618) WEXITSTATUS 0 Nov 05 01:48:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_949(8161618) done Nov 05 01:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_858(8161502) WEXITSTATUS 0 Nov 05 01:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_858(8161502) done Nov 05 01:48:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3669(8163019) in gpu on r808u08n01 Nov 05 01:48:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3670(8163020) in gpu on r808u17n01 Nov 05 01:48:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3671(8163021) in gpu on r4513u23n01 Nov 05 01:48:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154668 in day on r914u05n02 Nov 05 01:48:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154675 in day on r816u09n02 Nov 05 01:48:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1897(8163022) in day on r815u35n03 Nov 05 01:48:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1898(8163023) in day on r816u09n07 Nov 05 01:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_484(7572652) WEXITSTATUS 0 Nov 05 01:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_484(7572652) done Nov 05 01:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1052(7572018) WEXITSTATUS 0 Nov 05 01:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1052(7572018) done Nov 05 01:48:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1382(8135683) WEXITSTATUS 0 Nov 05 01:48:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1382(8135683) done Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6014(8163024) in scavenge on r912u05n04 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6015(8163025) in scavenge on r802u06n02 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6016(8163026) in scavenge on r802u11n02 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6017(8163027) in scavenge on r802u11n03 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6018(8163028) in scavenge on r816u23n12 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6019(8163029) in scavenge on r903u34n01 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6020(8163030) in scavenge on r907u28n01 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6021(8163031) in scavenge on r805u25n02 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6022(8163032) in scavenge on r913u24n04 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6023(8163033) in scavenge on r918u36n02 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6024(8163034) in scavenge on r815u29n04 Nov 05 01:48:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6025(8163035) in scavenge on r816u35n06 Nov 05 01:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_911(8161580) WEXITSTATUS 0 Nov 05 01:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_911(8161580) done Nov 05 01:49:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_904(8161573) WEXITSTATUS 0 Nov 05 01:49:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_904(8161573) done Nov 05 01:49:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5288(8158736) WEXITSTATUS 1 Nov 05 01:49:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5288(8158736) done Nov 05 01:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_909(8161578) WEXITSTATUS 0 Nov 05 01:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_909(8161578) done Nov 05 01:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_861(8161505) WEXITSTATUS 0 Nov 05 01:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_861(8161505) done Nov 05 01:49:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_912(8161581) WEXITSTATUS 0 Nov 05 01:49:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_912(8161581) done Nov 05 01:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_927(8161596) WEXITSTATUS 0 Nov 05 01:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_927(8161596) done Nov 05 01:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6021(8163031) WEXITSTATUS 1 Nov 05 01:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6021(8163031) done Nov 05 01:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_941(8161610) WEXITSTATUS 0 Nov 05 01:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_941(8161610) done Nov 05 01:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_944(8161613) WEXITSTATUS 0 Nov 05 01:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_944(8161613) done Nov 05 01:49:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_446(8160805) WEXITSTATUS 0 Nov 05 01:49:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_446(8160805) done Nov 05 01:49:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_478(8160837) WEXITSTATUS 0 Nov 05 01:49:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_478(8160837) done Nov 05 01:49:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_910(8161579) WEXITSTATUS 0 Nov 05 01:49:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_910(8161579) done Nov 05 01:49:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1272(8135221) WEXITSTATUS 0 Nov 05 01:49:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1272(8135221) done Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_903(8161572) WEXITSTATUS 0 Nov 05 01:49:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_903(8161572) done Nov 05 01:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_902(8161571) WEXITSTATUS 0 Nov 05 01:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_902(8161571) done Nov 05 01:49:41 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4471(8155788) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:49:41 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5943(8162938) has been requeued to reclaim resources for JobId=Invalid Nov 05 01:49:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4471(8155788) Nov 05 01:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_905(8161574) WEXITSTATUS 0 Nov 05 01:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_905(8161574) done Nov 05 01:49:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5943(8162938) Nov 05 01:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_906(8161575) WEXITSTATUS 0 Nov 05 01:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_906(8161575) done Nov 05 01:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5502(8160339) WEXITSTATUS 1 Nov 05 01:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5502(8160339) done Nov 05 01:49:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_867(8161511) WEXITSTATUS 0 Nov 05 01:49:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_867(8161511) done Nov 05 01:49:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5745(8162606) WEXITSTATUS 1 Nov 05 01:49:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5745(8162606) done Nov 05 01:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5550(8161040) WEXITSTATUS 1 Nov 05 01:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5550(8161040) done Nov 05 01:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1286(8135269) WEXITSTATUS 0 Nov 05 01:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1286(8135269) done Nov 05 01:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_921(8161590) WEXITSTATUS 0 Nov 05 01:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_921(8161590) done Nov 05 01:50:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3659(8162849) WEXITSTATUS 0 Nov 05 01:50:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3659(8162849) done Nov 05 01:50:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_826(7572995) Nov 05 01:50:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_13(8157698) Nov 05 01:50:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_907(8161576) WEXITSTATUS 0 Nov 05 01:50:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_907(8161576) done Nov 05 01:50:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_567(8161002) WEXITSTATUS 0 Nov 05 01:50:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_567(8161002) done Nov 05 01:50:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3669(8163019) WEXITSTATUS 0 Nov 05 01:50:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3669(8163019) done Nov 05 01:50:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_511(8160946) WEXITSTATUS 0 Nov 05 01:50:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_511(8160946) done Nov 05 01:50:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5956(8162952) WEXITSTATUS 1 Nov 05 01:50:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5956(8162952) done Nov 05 01:50:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_854(8161498) WEXITSTATUS 0 Nov 05 01:50:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_854(8161498) done Nov 05 01:50:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_864(8161508) WEXITSTATUS 0 Nov 05 01:50:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_864(8161508) done Nov 05 01:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_506(8160941) WEXITSTATUS 0 Nov 05 01:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_506(8160941) done Nov 05 01:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6006(8163011) WEXITSTATUS 1 Nov 05 01:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6006(8163011) done Nov 05 01:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5980(8162980) WEXITSTATUS 1 Nov 05 01:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5980(8162980) done Nov 05 01:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5606(8161693) WEXITSTATUS 1 Nov 05 01:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5606(8161693) done Nov 05 01:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5647(8161940) WEXITSTATUS 1 Nov 05 01:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5647(8161940) done Nov 05 01:50:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_555(8160990) WEXITSTATUS 0 Nov 05 01:50:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_555(8160990) done Nov 05 01:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5967(8162967) WEXITSTATUS 1 Nov 05 01:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5967(8162967) done Nov 05 01:51:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1476(8138477) WEXITSTATUS 0 Nov 05 01:51:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1476(8138477) done Nov 05 01:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2399(8152443) WEXITSTATUS 0 Nov 05 01:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2399(8152443) done Nov 05 01:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5959(8162955) WEXITSTATUS 1 Nov 05 01:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5959(8162955) done Nov 05 01:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3671(8163021) WEXITSTATUS 0 Nov 05 01:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3671(8163021) done Nov 05 01:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154635 WEXITSTATUS 0 Nov 05 01:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154635 done Nov 05 01:51:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_598(8161033) WEXITSTATUS 0 Nov 05 01:51:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_598(8161033) done Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:51:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:51:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1531(8145437) WEXITSTATUS 0 Nov 05 01:51:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1531(8145437) done Nov 05 01:51:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3672(8163036) in gpu on r808u08n01 Nov 05 01:51:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3673(8163037) in gpu on r808u11n01 Nov 05 01:51:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3674(8163038) in gpu on r4513u23n01 Nov 05 01:51:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154671 in day on r906u41n02 Nov 05 01:51:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154674 in day on r913u09n02 Nov 05 01:51:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154678 in day on r816u15n01 Nov 05 01:51:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1899(8163039) in day on r805u23n01 Nov 05 01:51:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1900(8163040) in day on r816u09n04 Nov 05 01:51:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1901(8163041) in day on r816u15n07 Nov 05 01:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6026(8163042) in scavenge on r816u03n10 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6027(8163043) in scavenge on r816u23n08 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6028(8163044) in scavenge on r816u29n10 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6029(8163045) in scavenge on r808u08n01 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6030(8163046) in scavenge on r913u09n01 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6031(8163047) in scavenge on r805u23n01 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6032(8163048) in scavenge on r805u25n02 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6033(8163049) in scavenge on r918u05n02 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6034(8163050) in scavenge on r918u09n02 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6035(8163051) in scavenge on r919u32n03 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6036(8163052) in scavenge on r816u15n01 Nov 05 01:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6037(8163053) in scavenge on r816u35n04 Nov 05 01:51:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_561(8160996) WEXITSTATUS 0 Nov 05 01:51:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_561(8160996) done Nov 05 01:51:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_709(8161304) WEXITSTATUS 0 Nov 05 01:51:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_709(8161304) done Nov 05 01:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6009(8163014) WEXITSTATUS 1 Nov 05 01:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6009(8163014) done Nov 05 01:51:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_628(8161139) WEXITSTATUS 0 Nov 05 01:51:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_628(8161139) done Nov 05 01:52:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8163054 NodeList=(null) usec=23099 Nov 05 01:52:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_587(8161022) WEXITSTATUS 0 Nov 05 01:52:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_587(8161022) done Nov 05 01:52:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4427(8155668) WEXITSTATUS 1 Nov 05 01:52:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4427(8155668) done Nov 05 01:52:34 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8163054 NodeList=r903u37n01 #CPUs=4 Partition=gpu_devel Nov 05 01:52:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6032(8163048) WEXITSTATUS 1 Nov 05 01:52:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6032(8163048) done Nov 05 01:52:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5588(8161551) WEXITSTATUS 1 Nov 05 01:52:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5588(8161551) done Nov 05 01:52:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_797(8115066) Nov 05 01:52:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154604 WEXITSTATUS 0 Nov 05 01:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154604 done Nov 05 01:53:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3672(8163036) WEXITSTATUS 0 Nov 05 01:53:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3672(8163036) done Nov 05 01:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1307(8135360) WEXITSTATUS 0 Nov 05 01:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1307(8135360) done Nov 05 01:53:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2252(8152169) WEXITSTATUS 0 Nov 05 01:53:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2252(8152169) done Nov 05 01:53:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_1(8162881) WEXITSTATUS 0 Nov 05 01:53:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_1(8162881) done Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:53:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8156102 Nov 05 01:53:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_1(8162887) WEXITSTATUS 0 Nov 05 01:53:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_1(8162887) done Nov 05 01:53:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1349(8135502) WEXITSTATUS 0 Nov 05 01:53:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1349(8135502) done Nov 05 01:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5836(8162803) WEXITSTATUS 1 Nov 05 01:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5836(8162803) done Nov 05 01:54:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154617 WEXITSTATUS 0 Nov 05 01:54:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154617 done Nov 05 01:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5970(8162970) WEXITSTATUS 1 Nov 05 01:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5970(8162970) done Nov 05 01:54:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1299(8135317) WEXITSTATUS 0 Nov 05 01:54:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1299(8135317) done Nov 05 01:54:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_29(8158661) WEXITSTATUS 0 Nov 05 01:54:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_29(8158661) done Nov 05 01:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3674(8163038) WEXITSTATUS 0 Nov 05 01:54:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3674(8163038) done Nov 05 01:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3644(8162627) WEXITSTATUS 0 Nov 05 01:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3644(8162627) done Nov 05 01:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3675(8163055) in gpu on r903u34n01 Nov 05 01:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3676(8163056) in gpu on r808u08n01 Nov 05 01:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3677(8163057) in gpu on r4513u23n01 Nov 05 01:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154676 in day on r913u01n03 Nov 05 01:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154677 in day on r913u05n03 Nov 05 01:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154681 in day on r816u09n06 Nov 05 01:54:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1902(8163058) in day on r815u29n03 Nov 05 01:54:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1903(8163059) in day on r816u09n05 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6038(8163060) in scavenge on r801u11n04 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6039(8163061) in scavenge on r802u32n03 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6040(8163062) in scavenge on r906u41n02 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6041(8163063) in scavenge on r914u01n01 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6042(8163064) in scavenge on r918u36n02 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6043(8163065) in scavenge on r805u02n02 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6044(8163066) in scavenge on r805u25n02 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6045(8163067) in scavenge on r913u01n03 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6046(8163068) in scavenge on r913u13n03 Nov 05 01:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6047(8163069) in scavenge on r815u09n04 Nov 05 01:54:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_860(8127974) in scavenge on r915u13n01 Nov 05 01:54:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6025(8163035) WEXITSTATUS 1 Nov 05 01:54:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6025(8163035) done Nov 05 01:55:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1356(8135533) WEXITSTATUS 0 Nov 05 01:55:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1356(8135533) done Nov 05 01:55:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_319(7572487) Nov 05 01:55:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_295(8114561) Nov 05 01:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5360(8159170) WEXITSTATUS 1 Nov 05 01:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5360(8159170) done Nov 05 01:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1420(8135953) WEXITSTATUS 0 Nov 05 01:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1420(8135953) done Nov 05 01:55:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4391(8155321) WEXITSTATUS 1 Nov 05 01:55:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4391(8155321) done Nov 05 01:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_17(8158591) WEXITSTATUS 0 Nov 05 01:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_17(8158591) done Nov 05 01:55:34 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144120 NodeList=r807u01n01,r914u09n03 #CPUs=72 Partition=scavenge Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:55:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5975(8162975) WEXITSTATUS 1 Nov 05 01:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5975(8162975) done Nov 05 01:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4684(8156346) WEXITSTATUS 1 Nov 05 01:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4684(8156346) done Nov 05 01:55:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1537(8146140) WEXITSTATUS 0 Nov 05 01:55:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1537(8146140) done Nov 05 01:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5744(8162605) WEXITSTATUS 1 Nov 05 01:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5744(8162605) done Nov 05 01:55:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8162614 uid 21828 Nov 05 01:55:58 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8162614 Nov 05 01:55:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21828 JobId=8162614 sig=9 returned: Invalid job id specified Nov 05 01:56:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8162747 uid 21828 Nov 05 01:56:07 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8162747 Nov 05 01:56:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21828 JobId=8162747 sig=9 returned: Invalid job id specified Nov 05 01:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5586(8161406) WEXITSTATUS 1 Nov 05 01:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5586(8161406) done Nov 05 01:56:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8162429 uid 21828 Nov 05 01:56:15 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8162429 Nov 05 01:56:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21828 JobId=8162429 sig=9 returned: Invalid job id specified Nov 05 01:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3643(8162590) WEXITSTATUS 0 Nov 05 01:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3643(8162590) done Nov 05 01:56:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8162429 uid 21828 Nov 05 01:56:23 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8162429 Nov 05 01:56:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21828 JobId=8162429 sig=9 returned: Invalid job id specified Nov 05 01:56:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5915(8162907) WEXITSTATUS 1 Nov 05 01:56:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5915(8162907) done Nov 05 01:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5733(8162566) WEXITSTATUS 1 Nov 05 01:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5733(8162566) done Nov 05 01:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1481(8138702) WEXITSTATUS 0 Nov 05 01:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1481(8138702) done Nov 05 01:56:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1389(8135780) WEXITSTATUS 0 Nov 05 01:56:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1389(8135780) done Nov 05 01:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5446(8159762) WEXITSTATUS 1 Nov 05 01:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5446(8159762) done Nov 05 01:56:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8162832 uid 21828 Nov 05 01:57:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_721(8161316) WEXITSTATUS 0 Nov 05 01:57:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_721(8161316) done Nov 05 01:57:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3678(8163070) in gpu on r4513u26n01 Nov 05 01:57:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154679 in day on r909u29n04 Nov 05 01:57:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154680 in day on r913u24n02 Nov 05 01:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 01:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_35(8158667) WEXITSTATUS 0 Nov 05 01:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_35(8158667) done Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6048(8163071) in scavenge on r801u30n01 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6049(8163072) in scavenge on r903u07n01 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6050(8163073) in scavenge on r911u05n02 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6051(8163074) in scavenge on r913u24n02 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6052(8163075) in scavenge on r913u24n02 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6053(8163076) in scavenge on r918u01n01 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6054(8163077) in scavenge on r918u01n03 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6055(8163078) in scavenge on r918u01n03 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6056(8163079) in scavenge on r918u01n03 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6057(8163080) in scavenge on r918u09n04 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6058(8163081) in scavenge on r918u09n04 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6059(8163082) in scavenge on r918u09n04 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6060(8163083) in scavenge on r918u09n04 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6061(8163084) in scavenge on r918u32n02 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6062(8163085) in scavenge on r919u36n01 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6063(8163086) in scavenge on r815u29n01 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6064(8163087) in scavenge on r816u09n03 Nov 05 01:57:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6065(8163088) in scavenge on r816u35n06 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:57:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:58:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2203(8126076) Nov 05 01:58:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5820(8162782) WEXITSTATUS 1 Nov 05 01:58:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5820(8162782) done Nov 05 01:58:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5702(8162364) WEXITSTATUS 1 Nov 05 01:58:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5702(8162364) done Nov 05 01:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6033(8163049) WEXITSTATUS 1 Nov 05 01:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6033(8163049) done Nov 05 01:58:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144121 NodeList=r915u32n04,r917u01n03 #CPUs=84 Partition=scavenge Nov 05 01:58:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 01:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_972(8161641) WEXITSTATUS 0 Nov 05 01:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_972(8161641) done Nov 05 01:58:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1470(8138471) WEXITSTATUS 0 Nov 05 01:58:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1470(8138471) done Nov 05 01:59:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_952(8161621) WEXITSTATUS 0 Nov 05 01:59:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_952(8161621) done Nov 05 01:59:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5120(8158453) WEXITSTATUS 1 Nov 05 01:59:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5120(8158453) done Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 01:59:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 01:59:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_14(8157788) Nov 05 01:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1561(8148684) WEXITSTATUS 0 Nov 05 01:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1561(8148684) done Nov 05 01:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154616 WEXITSTATUS 0 Nov 05 01:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154616 done Nov 05 02:00:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_817(8161461) WEXITSTATUS 0 Nov 05 02:00:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_817(8161461) done Nov 05 02:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 02:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 02:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 02:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 02:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6066(8163089) in scavenge on r815u29n12 Nov 05 02:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6067(8163090) in scavenge on r918u05n02 Nov 05 02:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6068(8163091) in scavenge on r816u03n05 Nov 05 02:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6069(8163092) in scavenge on r908u24n01 Nov 05 02:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 02:00:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133320 WEXITSTATUS 0 Nov 05 02:00:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133320 done Nov 05 02:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162531 WEXITSTATUS 0 Nov 05 02:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162531 done Nov 05 02:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3621(8161669) WEXITSTATUS 0 Nov 05 02:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3621(8161669) done Nov 05 02:00:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_15(8157790) Nov 05 02:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3678(8163070) WEXITSTATUS 0 Nov 05 02:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3678(8163070) done Nov 05 02:00:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5677(8162169) WEXITSTATUS 1 Nov 05 02:00:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5677(8162169) done Nov 05 02:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5782(8162732) WEXITSTATUS 1 Nov 05 02:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5782(8162732) done Nov 05 02:01:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2845(8128477) WEXITSTATUS 1 Nov 05 02:01:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2845(8128477) done Nov 05 02:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3673(8163037) WEXITSTATUS 0 Nov 05 02:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3673(8163037) done Nov 05 02:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5631(8161751) WEXITSTATUS 1 Nov 05 02:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5631(8161751) done Nov 05 02:01:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5263(8158697) WEXITSTATUS 1 Nov 05 02:01:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5263(8158697) done Nov 05 02:01:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5376(8159269) WEXITSTATUS 1 Nov 05 02:01:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5376(8159269) done Nov 05 02:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5714(8162427) WEXITSTATUS 1 Nov 05 02:01:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5714(8162427) done Nov 05 02:01:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5993(8162993) WEXITSTATUS 1 Nov 05 02:01:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5993(8162993) done Nov 05 02:01:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1367(8135643) WEXITSTATUS 0 Nov 05 02:01:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1367(8135643) done Nov 05 02:01:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5777(8162719) WEXITSTATUS 1 Nov 05 02:01:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5777(8162719) done Nov 05 02:01:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163093 InitPrio=21238 usec=3779 Nov 05 02:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5754(8162619) WEXITSTATUS 1 Nov 05 02:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5754(8162619) done Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:01:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:01:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_16(8157791) Nov 05 02:02:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_17(8157794) Nov 05 02:02:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_18(8157795) Nov 05 02:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5900(8162876) WEXITSTATUS 1 Nov 05 02:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5900(8162876) done Nov 05 02:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_1(8158574) WEXITSTATUS 0 Nov 05 02:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_1(8158574) done Nov 05 02:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 05 02:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n03 Nov 05 02:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162669 in day on r805u11n02 Nov 05 02:02:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3679(8163094) in gpu on r808u08n01 Nov 05 02:02:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3680(8163095) in gpu on r808u11n01 Nov 05 02:02:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3681(8163096) in gpu on r4513u26n01 Nov 05 02:02:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154682 in day on r909u04n02 Nov 05 02:02:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154684 in day on r816u15n06 Nov 05 02:02:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6070(8163097) in scavenge on r816u09n10 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6071(8163098) in scavenge on r808u08n01 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6072(8163099) in scavenge on r805u25n02 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6073(8163100) in scavenge on r806u06n02 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6074(8163101) in scavenge on r806u06n02 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6075(8163102) in scavenge on r806u06n02 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6076(8163103) in scavenge on r904u10n03 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6077(8163104) in scavenge on r919u01n04 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6078(8163105) in scavenge on r816u03n04 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6079(8163106) in scavenge on r816u03n07 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6080(8163107) in scavenge on r816u29n03 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6081(8163108) in scavenge on r816u35n06 Nov 05 02:02:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6082(8163109) in scavenge on r805u34n01 Nov 05 02:02:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8156198 Nov 05 02:02:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_19(8157813) Nov 05 02:02:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_6(8158580) WEXITSTATUS 0 Nov 05 02:02:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_6(8158580) done Nov 05 02:02:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_913(8161582) WEXITSTATUS 0 Nov 05 02:02:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_913(8161582) done Nov 05 02:02:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1256(8135159) WEXITSTATUS 0 Nov 05 02:02:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1256(8135159) done Nov 05 02:02:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 02:02:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:02:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 02:03:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3679(8163094) WEXITSTATUS 0 Nov 05 02:03:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3679(8163094) done Nov 05 02:03:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_564(7572733) Nov 05 02:03:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_704(7572873) Nov 05 02:03:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_706(7572875) Nov 05 02:03:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2210(8126087) Nov 05 02:03:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_20(8157815) Nov 05 02:03:11 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5784(8162734) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:03:11 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5785(8162735) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:03:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6044(8163066) WEXITSTATUS 1 Nov 05 02:03:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6044(8163066) done Nov 05 02:03:13 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5784(8162734) Nov 05 02:03:13 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5785(8162735) Nov 05 02:03:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3680(8163095) WEXITSTATUS 0 Nov 05 02:03:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3680(8163095) done Nov 05 02:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5111(8158444) WEXITSTATUS 1 Nov 05 02:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5111(8158444) done Nov 05 02:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_14(8158588) WEXITSTATUS 0 Nov 05 02:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_14(8158588) done Nov 05 02:03:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_21(8157817) Nov 05 02:03:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_22(8157818) Nov 05 02:03:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_23(8157819) Nov 05 02:03:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_24(8157820) Nov 05 02:03:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:03:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3666(8162995) WEXITSTATUS 0 Nov 05 02:03:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3666(8162995) done Nov 05 02:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_23(8158622) WEXITSTATUS 0 Nov 05 02:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_23(8158622) done Nov 05 02:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5890(8162866) WEXITSTATUS 1 Nov 05 02:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5890(8162866) done Nov 05 02:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6047(8163069) WEXITSTATUS 1 Nov 05 02:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6047(8163069) done Nov 05 02:04:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1355(8135532) WEXITSTATUS 0 Nov 05 02:04:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1355(8135532) done Nov 05 02:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154614 WEXITSTATUS 0 Nov 05 02:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154614 done Nov 05 02:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154654 WEXITSTATUS 0 Nov 05 02:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154654 done Nov 05 02:04:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6035(8163051) WEXITSTATUS 1 Nov 05 02:04:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6035(8163051) done Nov 05 02:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_37(8158685) WEXITSTATUS 0 Nov 05 02:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_37(8158685) done Nov 05 02:04:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1334(8135467) WEXITSTATUS 0 Nov 05 02:04:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1334(8135467) done Nov 05 02:04:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144122 NodeList=r908u22n02,r916u01n04 #CPUs=72 Partition=scavenge Nov 05 02:04:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_25(8157823) Nov 05 02:04:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_957(8128298) WEXITSTATUS 0 Nov 05 02:04:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_957(8128298) done Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:04:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6046(8163068) WEXITSTATUS 1 Nov 05 02:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6046(8163068) done Nov 05 02:05:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4565(8156022) WEXITSTATUS 1 Nov 05 02:05:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4565(8156022) done Nov 05 02:05:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_26(8157824) Nov 05 02:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3681(8163096) WEXITSTATUS 0 Nov 05 02:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3681(8163096) done Nov 05 02:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6051(8163074) WEXITSTATUS 1 Nov 05 02:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6051(8163074) done Nov 05 02:05:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3682(8163110) in gpu on r808u08n01 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3683(8163111) in gpu on r808u11n01 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3684(8163112) in gpu on r808u14n01 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3685(8163113) in gpu on r4513u26n01 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154683 in day on r909u01n01 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154685 in day on r913u13n02 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154687 in day on r815u29n04 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154688 in day on r816u03n05 Nov 05 02:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154689 in day on r816u09n07 Nov 05 02:05:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1904(8163114) in day on r912u32n01 Nov 05 02:05:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1905(8163115) in day on r913u24n03 Nov 05 02:05:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6083(8163116) in scavenge on r908u39n04 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6084(8163117) in scavenge on r908u39n04 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6085(8163118) in scavenge on r908u33n01 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6086(8163119) in scavenge on r908u33n01 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6087(8163120) in scavenge on r908u33n04 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6088(8163121) in scavenge on r908u33n04 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6089(8163122) in scavenge on r805u25n02 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6090(8163123) in scavenge on r914u01n01 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6091(8163124) in scavenge on r918u05n03 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6092(8163125) in scavenge on r918u05n03 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6093(8163126) in scavenge on r919u32n03 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6094(8163127) in scavenge on r912u36n02 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6095(8163128) in scavenge on r913u13n03 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6096(8163129) in scavenge on r913u24n02 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6097(8163130) in scavenge on r913u24n03 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6098(8163131) in scavenge on r815u09n04 Nov 05 02:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6099(8163132) in scavenge on r816u29n03 Nov 05 02:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4505(8155888) WEXITSTATUS 1 Nov 05 02:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4505(8155888) done Nov 05 02:05:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 02:05:36 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 02:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6089(8163122) WEXITSTATUS 1 Nov 05 02:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6089(8163122) done Nov 05 02:05:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2213(8126096) Nov 05 02:05:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5225(8158612) WEXITSTATUS 1 Nov 05 02:05:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5225(8158612) done Nov 05 02:05:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159905 WEXITSTATUS 0 Nov 05 02:05:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159905 done Nov 05 02:06:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_13(8158587) WEXITSTATUS 0 Nov 05 02:06:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_13(8158587) done Nov 05 02:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161550 WEXITSTATUS 0 Nov 05 02:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161550 done Nov 05 02:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161702 WEXITSTATUS 0 Nov 05 02:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161702 done Nov 05 02:06:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3683(8163111) WEXITSTATUS 0 Nov 05 02:06:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3683(8163111) done Nov 05 02:06:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_27(8157835) Nov 05 02:06:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6096(8163129) WEXITSTATUS 1 Nov 05 02:06:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6096(8163129) done Nov 05 02:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154649 WEXITSTATUS 0 Nov 05 02:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154649 done Nov 05 02:06:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6088(8163121) WEXITSTATUS 1 Nov 05 02:06:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6088(8163121) done Nov 05 02:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907437 WEXITSTATUS 0 Nov 05 02:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907437 done Nov 05 02:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1298(8135316) WEXITSTATUS 0 Nov 05 02:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1298(8135316) done Nov 05 02:06:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_28(8157837) Nov 05 02:06:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_29(8157838) Nov 05 02:06:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1527(8145317) WEXITSTATUS 0 Nov 05 02:06:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1527(8145317) done Nov 05 02:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6071(8163098) WEXITSTATUS 1 Nov 05 02:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6071(8163098) done Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:06:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:07:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1459(8138136) WEXITSTATUS 0 Nov 05 02:07:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1459(8138136) done Nov 05 02:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3670(8163020) WEXITSTATUS 0 Nov 05 02:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3670(8163020) done Nov 05 02:07:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158107 WEXITSTATUS 0 Nov 05 02:07:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158107 done Nov 05 02:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1366(8135642) WEXITSTATUS 0 Nov 05 02:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1366(8135642) done Nov 05 02:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5750(8162615) WEXITSTATUS 1 Nov 05 02:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5750(8162615) done Nov 05 02:07:34 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144124 NodeList=r807u39n01,r914u09n04 #CPUs=68 Partition=scavenge Nov 05 02:07:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3684(8163112) WEXITSTATUS 0 Nov 05 02:07:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3684(8163112) done Nov 05 02:07:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5854(8162825) WEXITSTATUS 1 Nov 05 02:07:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5854(8162825) done Nov 05 02:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6020(8163030) WEXITSTATUS 1 Nov 05 02:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6020(8163030) done Nov 05 02:07:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3675(8163055) WEXITSTATUS 0 Nov 05 02:07:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3675(8163055) done Nov 05 02:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4909(8158059) WEXITSTATUS 1 Nov 05 02:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4909(8158059) done Nov 05 02:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1345(8135490) WEXITSTATUS 0 Nov 05 02:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1345(8135490) done Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162670 in day on r919u36n02 Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163093 in day on r909u29n03,r913u28n01 Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907438 in gpu on r805u38n01 Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3686(8163133) in gpu on r903u34n01 Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3687(8163134) in gpu on r808u11n01 Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3688(8163135) in gpu on r808u14n01 Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3689(8163136) in gpu on r808u17n01 Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154686 in day on r909u01n04 Nov 05 02:08:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154692 in day on r816u09n04 Nov 05 02:08:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1906(8163137) in day on r912u36n04 Nov 05 02:08:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_30(8157840) Nov 05 02:08:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6100(8163138) in scavenge on r907u05n03 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6101(8163139) in scavenge on r908u37n03 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6102(8163140) in scavenge on r908u37n03 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6103(8163141) in scavenge on r908u33n04 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6104(8163142) in scavenge on r802u11n03 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6105(8163143) in scavenge on r808u08n01 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6106(8163144) in scavenge on r907u28n01 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6107(8163145) in scavenge on r805u25n02 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6108(8163146) in scavenge on r807u05n04 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6109(8163147) in scavenge on r912u28n04 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6110(8163148) in scavenge on r912u36n04 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6111(8163149) in scavenge on r913u24n02 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6112(8163150) in scavenge on r913u28n04 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6113(8163151) in scavenge on r906u32n01 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3696(8138600) WEXITSTATUS 1 Nov 05 02:08:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3696(8138600) done Nov 05 02:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5516(8160519) WEXITSTATUS 1 Nov 05 02:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5516(8160519) done Nov 05 02:08:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3689(8163136) WEXITSTATUS 0 Nov 05 02:08:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3689(8163136) done Nov 05 02:08:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1295(8135308) WEXITSTATUS 0 Nov 05 02:08:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1295(8135308) done Nov 05 02:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 02:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 02:08:28 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 02:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154650 WEXITSTATUS 0 Nov 05 02:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154650 done Nov 05 02:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1360(8135623) WEXITSTATUS 0 Nov 05 02:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1360(8135623) done Nov 05 02:08:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1300(8135324) WEXITSTATUS 0 Nov 05 02:08:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1300(8135324) done Nov 05 02:08:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_31(8157858) Nov 05 02:08:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6066(8163089) WEXITSTATUS 1 Nov 05 02:08:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6066(8163089) done Nov 05 02:08:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163093 WEXITSTATUS 1 Nov 05 02:08:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163093 done Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:08:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1384(8135685) WEXITSTATUS 0 Nov 05 02:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1384(8135685) done Nov 05 02:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1560(8148683) WEXITSTATUS 0 Nov 05 02:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1560(8148683) done Nov 05 02:09:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3688(8163135) WEXITSTATUS 0 Nov 05 02:09:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3688(8163135) done Nov 05 02:09:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_32(8157859) Nov 05 02:09:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:09:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_354(8160647) WEXITSTATUS 0 Nov 05 02:09:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_354(8160647) done Nov 05 02:09:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_11(8158585) WEXITSTATUS 0 Nov 05 02:09:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_11(8158585) done Nov 05 02:09:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4222(8152277) WEXITSTATUS 1 Nov 05 02:09:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4222(8152277) done Nov 05 02:09:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_33(8157860) Nov 05 02:09:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_34(8157861) Nov 05 02:09:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_35(8157862) Nov 05 02:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_351(8160644) WEXITSTATUS 0 Nov 05 02:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_351(8160644) done Nov 05 02:09:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4738(8156978) WEXITSTATUS 1 Nov 05 02:09:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4738(8156978) done Nov 05 02:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5792(8162751) WEXITSTATUS 1 Nov 05 02:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5792(8162751) done Nov 05 02:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_352(8160645) WEXITSTATUS 0 Nov 05 02:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_352(8160645) done Nov 05 02:10:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1339(8135477) WEXITSTATUS 0 Nov 05 02:10:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1339(8135477) done Nov 05 02:10:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159917 WEXITSTATUS 0 Nov 05 02:10:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159917 done Nov 05 02:10:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:10:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:10:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:10:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:10:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:10:33 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144125 NodeList=r914u13n[03-04] #CPUs=72 Partition=scavenge Nov 05 02:10:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5995(8163000) WEXITSTATUS 1 Nov 05 02:10:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5995(8163000) done Nov 05 02:10:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_573(7572742) Nov 05 02:10:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_728(7572897) Nov 05 02:10:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_36(8157865) Nov 05 02:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6068(8163091) WEXITSTATUS 1 Nov 05 02:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6068(8163091) done Nov 05 02:10:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_355(8160648) WEXITSTATUS 0 Nov 05 02:10:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_355(8160648) done Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:10:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162671 in day on r806u02n02 Nov 05 02:10:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1297(8135315) WEXITSTATUS 0 Nov 05 02:10:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1297(8135315) done Nov 05 02:10:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3690(8163152) in gpu on r808u14n01 Nov 05 02:10:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3691(8163153) in gpu on r808u17n01 Nov 05 02:10:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154690 in day on r909u07n03 Nov 05 02:10:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154691 in day on r918u09n01 Nov 05 02:10:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154693 in day on r912u28n03 Nov 05 02:10:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154694 in day on r913u01n01 Nov 05 02:10:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154695 in day on r913u28n01 Nov 05 02:11:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1907(8163154) in day on r913u24n01 Nov 05 02:11:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_210(7572378) WEXITSTATUS 0 Nov 05 02:11:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_210(7572378) done Nov 05 02:11:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6114(8163155) in scavenge on r906u01n02 Nov 05 02:11:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6115(8163156) in scavenge on r908u35n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6116(8163157) in scavenge on r908u35n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6117(8163158) in scavenge on r908u37n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6118(8163159) in scavenge on r908u37n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6119(8163160) in scavenge on r801u23n01 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6120(8163161) in scavenge on r911u01n01 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6121(8163162) in scavenge on r815u09n11 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6122(8163163) in scavenge on r815u29n12 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6123(8163164) in scavenge on r912u28n03 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6124(8163165) in scavenge on r912u28n03 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6125(8163166) in scavenge on r913u01n01 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6126(8163167) in scavenge on r913u13n02 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6127(8163168) in scavenge on r913u24n01 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6128(8163169) in scavenge on r913u28n01 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6129(8163170) in scavenge on r913u28n01 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6130(8163171) in scavenge on r909u01n03 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6131(8163172) in scavenge on r909u04n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6132(8163173) in scavenge on r909u04n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6133(8163174) in scavenge on r909u04n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6134(8163175) in scavenge on r909u07n02 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6135(8163176) in scavenge on r909u07n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6136(8163177) in scavenge on r909u07n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6137(8163178) in scavenge on r909u10n01 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6138(8163179) in scavenge on r909u10n01 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6139(8163180) in scavenge on r909u10n03 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6140(8163181) in scavenge on r909u13n02 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6141(8163182) in scavenge on r909u13n02 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6142(8163183) in scavenge on r909u13n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6143(8163184) in scavenge on r909u13n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6144(8163185) in scavenge on r909u13n04 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6145(8163186) in scavenge on r909u29n03 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_363(7572531) in scavenge on r801u06n02 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_370(7572538) in scavenge on r815u15n11 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_404(7572572) in scavenge on r816u23n07 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_498(7572666) in scavenge on r816u23n12 Nov 05 02:11:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_353(7572521) in scavenge on r909u29n03 Nov 05 02:11:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4737(8156977) WEXITSTATUS 1 Nov 05 02:11:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4737(8156977) done Nov 05 02:11:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1512(8143571) in scavenge on r816u35n10 Nov 05 02:11:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1120(8131612) in scavenge on r816u35n10 Nov 05 02:11:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1436(8135969) in scavenge on r815u15n09 Nov 05 02:11:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1456(8137385) in scavenge on r815u15n09 Nov 05 02:11:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_37(8157867) Nov 05 02:11:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1371(8135658) WEXITSTATUS 0 Nov 05 02:11:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1371(8135658) done Nov 05 02:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2935(8128619) WEXITSTATUS 1 Nov 05 02:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2935(8128619) done Nov 05 02:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3691(8163153) WEXITSTATUS 0 Nov 05 02:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3691(8163153) done Nov 05 02:11:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_38(8157881) Nov 05 02:11:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5285(8158733) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:11:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5712(8162425) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:11:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5828(8162790) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5796(8162755) WEXITSTATUS 1 Nov 05 02:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5796(8162755) done Nov 05 02:11:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5828(8162790) Nov 05 02:11:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5285(8158733) Nov 05 02:11:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5712(8162425) Nov 05 02:11:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5434(8159558) WEXITSTATUS 1 Nov 05 02:11:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5434(8159558) done Nov 05 02:12:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6135(8163176) WEXITSTATUS 1 Nov 05 02:12:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6135(8163176) done Nov 05 02:12:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6060(8163083) WEXITSTATUS 1 Nov 05 02:12:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6060(8163083) done Nov 05 02:12:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5998(8163003) WEXITSTATUS 1 Nov 05 02:12:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5998(8163003) done Nov 05 02:12:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_39(8157882) Nov 05 02:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3654(8162794) WEXITSTATUS 0 Nov 05 02:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3654(8162794) done Nov 05 02:12:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154636 WEXITSTATUS 0 Nov 05 02:12:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154636 done Nov 05 02:12:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_974(8128351) WEXITSTATUS 0 Nov 05 02:12:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_974(8128351) done Nov 05 02:12:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3687(8163134) WEXITSTATUS 0 Nov 05 02:12:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3687(8163134) done Nov 05 02:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5514(8160517) WEXITSTATUS 1 Nov 05 02:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5514(8160517) done Nov 05 02:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4470(8155787) WEXITSTATUS 1 Nov 05 02:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4470(8155787) done Nov 05 02:12:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_40(8157886) Nov 05 02:12:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_41(8157887) Nov 05 02:12:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_42(8157888) Nov 05 02:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5197(8158560) WEXITSTATUS 1 Nov 05 02:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5197(8158560) done Nov 05 02:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5659(8162026) WEXITSTATUS 1 Nov 05 02:12:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5659(8162026) done Nov 05 02:12:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5936(8162931) WEXITSTATUS 1 Nov 05 02:12:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5936(8162931) done Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:12:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:13:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6085(8163118) WEXITSTATUS 1 Nov 05 02:13:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6085(8163118) done Nov 05 02:13:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_353(8160646) WEXITSTATUS 0 Nov 05 02:13:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_353(8160646) done Nov 05 02:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1509(8143365) WEXITSTATUS 0 Nov 05 02:13:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1509(8143365) done Nov 05 02:13:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3682(8163110) WEXITSTATUS 0 Nov 05 02:13:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3682(8163110) done Nov 05 02:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1390(8135799) WEXITSTATUS 0 Nov 05 02:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1390(8135799) done Nov 05 02:13:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144126 NodeList=r915u29n[01,04] #CPUs=96 Partition=scavenge Nov 05 02:13:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3692(8163187) in gpu on r808u08n01 Nov 05 02:13:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3693(8163188) in gpu on r808u08n01 Nov 05 02:13:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3694(8163189) in gpu on r808u11n01 Nov 05 02:13:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3695(8163190) in gpu on r808u17n01 Nov 05 02:13:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154696 in day on r805u02n04 Nov 05 02:13:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154697 in day on r805u14n01 Nov 05 02:13:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154700 in day on r815u29n02 Nov 05 02:13:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154701 in day on r816u09n06 Nov 05 02:13:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1908(8163191) in day on r806u06n01 Nov 05 02:13:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1909(8163192) in day on r909u10n04 Nov 05 02:13:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_802(7572971) Nov 05 02:13:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_837(7573006) Nov 05 02:13:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_43(8157923) Nov 05 02:13:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6146(8163193) in scavenge on r908u37n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6147(8163194) in scavenge on r908u37n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6148(8163195) in scavenge on r908u33n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6149(8163196) in scavenge on r816u29n08 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6150(8163197) in scavenge on r816u35n09 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6151(8163198) in scavenge on r906u41n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6152(8163199) in scavenge on r805u06n04 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6153(8163200) in scavenge on r805u14n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6154(8163201) in scavenge on r805u14n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6155(8163202) in scavenge on r912u28n04 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6156(8163203) in scavenge on r913u28n02 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6157(8163204) in scavenge on r914u01n02 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6158(8163205) in scavenge on r914u05n02 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6159(8163206) in scavenge on r806u02n03 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6160(8163207) in scavenge on r806u04n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6161(8163208) in scavenge on r806u04n03 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_425(7572593) in scavenge on r919u09n04 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_483(7572651) in scavenge on r919u32n04 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_347(8114614) in scavenge on r806u04n03 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_985(7571951) in scavenge on r806u06n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_368(7572536) in scavenge on r806u14n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_400(7572568) in scavenge on r909u04n01 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_426(7572594) in scavenge on r909u04n03 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_492(7572660) in scavenge on r909u10n02 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_401(7572569) in scavenge on r815u23n05 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_419(7572587) in scavenge on r815u23n05 Nov 05 02:13:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_461(7572629) in scavenge on r816u15n02 Nov 05 02:13:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6150(8163197) WEXITSTATUS 1 Nov 05 02:13:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6150(8163197) done Nov 05 02:13:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5737(8162593) WEXITSTATUS 1 Nov 05 02:13:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5737(8162593) done Nov 05 02:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_22(8158621) WEXITSTATUS 0 Nov 05 02:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_22(8158621) done Nov 05 02:14:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5673(8162165) WEXITSTATUS 1 Nov 05 02:14:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5673(8162165) done Nov 05 02:14:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1365(8135637) WEXITSTATUS 0 Nov 05 02:14:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1365(8135637) done Nov 05 02:14:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5660(8162027) WEXITSTATUS 1 Nov 05 02:14:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5660(8162027) done Nov 05 02:14:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163209 InitPrio=22669 usec=1928 Nov 05 02:14:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163210 InitPrio=22669 usec=1454 Nov 05 02:14:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163211 InitPrio=22669 usec=1514 Nov 05 02:14:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163212 InitPrio=22669 usec=1242 Nov 05 02:14:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163213 InitPrio=22669 usec=2276 Nov 05 02:14:14 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_44(8157933) Nov 05 02:14:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3693(8163188) WEXITSTATUS 0 Nov 05 02:14:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3693(8163188) done Nov 05 02:14:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163214 InitPrio=22669 usec=2333 Nov 05 02:14:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163215 InitPrio=22669 usec=1635 Nov 05 02:14:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163216 InitPrio=22669 usec=1451 Nov 05 02:14:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163217 InitPrio=22669 usec=1563 Nov 05 02:14:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163218 InitPrio=22669 usec=1527 Nov 05 02:14:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163219 InitPrio=22669 usec=2188 Nov 05 02:14:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163220 InitPrio=22669 usec=1758 Nov 05 02:14:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163221 InitPrio=22669 usec=2562 Nov 05 02:14:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144128 NodeList=r914u13n02,r916u01n03 #CPUs=84 Partition=scavenge Nov 05 02:14:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163222 InitPrio=22669 usec=1523 Nov 05 02:14:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163223 InitPrio=22669 usec=2759 Nov 05 02:14:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3648(8162740) WEXITSTATUS 0 Nov 05 02:14:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3648(8162740) done Nov 05 02:14:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_45(8157945) Nov 05 02:14:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:14:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:14:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1290(8135280) WEXITSTATUS 0 Nov 05 02:14:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1290(8135280) done Nov 05 02:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133194 WEXITSTATUS 0 Nov 05 02:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133194 done Nov 05 02:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6114(8163155) WEXITSTATUS 1 Nov 05 02:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6114(8163155) done Nov 05 02:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1469(8138470) WEXITSTATUS 0 Nov 05 02:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1469(8138470) done Nov 05 02:15:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3695(8163190) WEXITSTATUS 0 Nov 05 02:15:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3695(8163190) done Nov 05 02:15:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1393(8135802) WEXITSTATUS 0 Nov 05 02:15:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1393(8135802) done Nov 05 02:15:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_46(8157947) Nov 05 02:15:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_47(8157948) Nov 05 02:15:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_48(8157949) Nov 05 02:15:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163224 InitPrio=21238 usec=2576 Nov 05 02:15:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_50(8161808) WEXITSTATUS 0 Nov 05 02:15:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_50(8161808) done Nov 05 02:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3692(8163187) WEXITSTATUS 0 Nov 05 02:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3692(8163187) done Nov 05 02:15:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:53 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:54 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:54 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:56 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:57 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:57 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:58 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:15:59 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:00 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:00 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:01 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:02 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:04 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:05 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:08 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:09 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:11 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:16:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_890(7573059) Nov 05 02:16:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_897(7573066) Nov 05 02:16:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5911(8162903) WEXITSTATUS 1 Nov 05 02:16:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5911(8162903) done Nov 05 02:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_2(8162882) WEXITSTATUS 0 Nov 05 02:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_2(8162882) done Nov 05 02:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_5(8163225) in pi_ohern on r815u03n12 Nov 05 02:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3696(8163226) in gpu on r903u40n01 Nov 05 02:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3697(8163227) in gpu on r808u08n01 Nov 05 02:16:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3698(8163228) in gpu on r808u08n01 Nov 05 02:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3699(8163229) in gpu on r808u17n01 Nov 05 02:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1910(8163230) in day on r909u10n03 Nov 05 02:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1911(8163231) in day on r914u01n03 Nov 05 02:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1912(8163232) in day on r918u05n03 Nov 05 02:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1913(8163233) in day on r815u23n04 Nov 05 02:16:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1914(8163234) in day on r815u23n04 Nov 05 02:16:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163235 InitPrio=22669 usec=2606 Nov 05 02:16:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163236 InitPrio=22669 usec=1871 Nov 05 02:16:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163237 InitPrio=22669 usec=1490 Nov 05 02:16:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163238 InitPrio=22669 usec=1484 Nov 05 02:16:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163239 InitPrio=22669 usec=1430 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163209 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163210 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163211 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163212 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163213 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163214 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163215 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163216 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163217 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163218 in scavenge on r906u01n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163219 in scavenge on r801u30n01 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163220 in scavenge on r801u30n01 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163221 in scavenge on r801u30n01 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163222 in scavenge on r801u30n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163223 in scavenge on r801u30n03 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6162(8163240) in scavenge on r908u37n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6163(8163241) in scavenge on r908u39n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6164(8163242) in scavenge on r908u39n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6165(8163243) in scavenge on r908u39n03 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6166(8163244) in scavenge on r908u39n03 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6167(8163245) in scavenge on r802u09n03 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_493(7572661) in scavenge on r816u23n08 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_783(7572952) in scavenge on r816u35n09 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_494(7572662) in scavenge on r907u24n01 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_495(7572663) in scavenge on r805u06n01 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_642(8114911) in scavenge on r805u06n01 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_857(7573026) in scavenge on r913u09n02 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_74(8114340) in scavenge on r913u09n03 Nov 05 02:16:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_563(8114830) in scavenge on r815u23n04 Nov 05 02:16:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144129 NodeList=r915u29n[02-03] #CPUs=96 Partition=scavenge Nov 05 02:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1586(8151846) WEXITSTATUS 0 Nov 05 02:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1586(8151846) done Nov 05 02:16:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8149759_49(8149759) Nov 05 02:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163054 WEXITSTATUS 130 Nov 05 02:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163054 done Nov 05 02:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1353(8135530) WEXITSTATUS 0 Nov 05 02:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1353(8135530) done Nov 05 02:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6148(8163195) WEXITSTATUS 1 Nov 05 02:16:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6148(8163195) done Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:16:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1316(8135393) WEXITSTATUS 0 Nov 05 02:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1316(8135393) done Nov 05 02:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6138(8163179) WEXITSTATUS 1 Nov 05 02:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6138(8163179) done Nov 05 02:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1344(8135489) WEXITSTATUS 0 Nov 05 02:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1344(8135489) done Nov 05 02:17:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1327(8135424) WEXITSTATUS 0 Nov 05 02:17:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1327(8135424) done Nov 05 02:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5208(8158571) WEXITSTATUS 1 Nov 05 02:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5208(8158571) done Nov 05 02:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_2(8162888) WEXITSTATUS 0 Nov 05 02:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_2(8162888) done Nov 05 02:17:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5951(8162947) WEXITSTATUS 1 Nov 05 02:17:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5951(8162947) done Nov 05 02:17:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_51(8161809) WEXITSTATUS 0 Nov 05 02:17:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_51(8161809) done Nov 05 02:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6105(8163143) WEXITSTATUS 1 Nov 05 02:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6105(8163143) done Nov 05 02:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1588(8151953) WEXITSTATUS 0 Nov 05 02:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1588(8151953) done Nov 05 02:17:50 r912u23n01 slurmctld[163760]: slurmctld: Job specs has invalid feature list: V100 Nov 05 02:17:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Invalid feature specification Nov 05 02:17:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_21(8158614) WEXITSTATUS 0 Nov 05 02:17:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_21(8158614) done Nov 05 02:17:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_3(8162883) WEXITSTATUS 0 Nov 05 02:17:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_3(8162883) done Nov 05 02:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3543(8158546) WEXITSTATUS 0 Nov 05 02:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3543(8158546) done Nov 05 02:17:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1579(8150822) WEXITSTATUS 0 Nov 05 02:17:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1579(8150822) done Nov 05 02:18:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8163246 NodeList=(null) usec=26525 Nov 05 02:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5952(8162948) WEXITSTATUS 1 Nov 05 02:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5952(8162948) done Nov 05 02:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162528 WEXITSTATUS 0 Nov 05 02:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162528 done Nov 05 02:18:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:18:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:18:25 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1446(8136305) WEXITSTATUS 0 Nov 05 02:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1446(8136305) done Nov 05 02:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154640 WEXITSTATUS 0 Nov 05 02:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154640 done Nov 05 02:18:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8163246 NodeList=r805u30n01 #CPUs=5 Partition=gpu_devel Nov 05 02:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1380(8135681) WEXITSTATUS 0 Nov 05 02:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1380(8135681) done Nov 05 02:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4926(8158113) WEXITSTATUS 1 Nov 05 02:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4926(8158113) done Nov 05 02:18:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2227(8126123) Nov 05 02:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_350(8160643) WEXITSTATUS 0 Nov 05 02:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_350(8160643) done Nov 05 02:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4240(8152907) WEXITSTATUS 1 Nov 05 02:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4240(8152907) done Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:18:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1362(8135634) WEXITSTATUS 0 Nov 05 02:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1362(8135634) done Nov 05 02:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5663(8162030) WEXITSTATUS 1 Nov 05 02:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5663(8162030) done Nov 05 02:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1323(8135420) WEXITSTATUS 0 Nov 05 02:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1323(8135420) done Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_6(8163247) in pi_ohern on r815u03n12 Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3700(8163248) in gpu on r4513u29n01 Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154698 in day on r805u11n03 Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154651 WEXITSTATUS 0 Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154651 done Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6083(8163116) WEXITSTATUS 1 Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6083(8163116) done Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5904(8162896) WEXITSTATUS 1 Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5904(8162896) done Nov 05 02:19:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154704 in day on r904u07n01 Nov 05 02:19:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1915(8163249) in day on r806u14n04 Nov 05 02:19:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1916(8163250) in day on r807u03n04 Nov 05 02:19:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1917(8163251) in day on r906u41n01 Nov 05 02:19:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1918(8163252) in day on r904u04n04 Nov 05 02:19:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1919(8163253) in day on r816u15n02 Nov 05 02:19:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:19:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5725(8162527) WEXITSTATUS 1 Nov 05 02:19:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5725(8162527) done Nov 05 02:19:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128436 WEXITSTATUS 0 Nov 05 02:19:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128436 done Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163235 in scavenge on r801u30n03 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163236 in scavenge on r801u30n03 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163237 in scavenge on r801u30n03 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163238 in scavenge on r801u30n04 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163239 in scavenge on r801u30n04 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6168(8163254) in scavenge on r908u35n03 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6169(8163255) in scavenge on r908u35n03 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6170(8163256) in scavenge on r908u39n01 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6171(8163257) in scavenge on r908u39n01 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6172(8163258) in scavenge on r908u33n01 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6173(8163259) in scavenge on r801u23n03 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6174(8163260) in scavenge on r911u05n02 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6175(8163261) in scavenge on r915u35n04 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6176(8163262) in scavenge on r808u08n01 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6177(8163263) in scavenge on r808u11n01 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6178(8163264) in scavenge on r4513u23n01 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6179(8163265) in scavenge on r904u07n01 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6180(8163266) in scavenge on r904u04n04 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6181(8163267) in scavenge on r918u09n01 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_610(8114877) in scavenge on r816u29n04 Nov 05 02:19:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_367(7572535) in scavenge on r806u31n01 Nov 05 02:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_349(8160642) WEXITSTATUS 0 Nov 05 02:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_349(8160642) done Nov 05 02:19:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_3(8162889) WEXITSTATUS 0 Nov 05 02:19:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_3(8162889) done Nov 05 02:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5859(8162830) WEXITSTATUS 1 Nov 05 02:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5859(8162830) done Nov 05 02:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3698(8163228) WEXITSTATUS 0 Nov 05 02:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3698(8163228) done Nov 05 02:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159920 WEXITSTATUS 0 Nov 05 02:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159920 done Nov 05 02:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_53(8161811) WEXITSTATUS 0 Nov 05 02:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_53(8161811) done Nov 05 02:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1396(8135805) WEXITSTATUS 0 Nov 05 02:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1396(8135805) done Nov 05 02:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3696(8163226) WEXITSTATUS 0 Nov 05 02:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3696(8163226) done Nov 05 02:19:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133209 WEXITSTATUS 0 Nov 05 02:19:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133209 done Nov 05 02:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3681(8138257) WEXITSTATUS 1 Nov 05 02:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3681(8138257) done Nov 05 02:20:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1440(8135973) WEXITSTATUS 0 Nov 05 02:20:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1440(8135973) done Nov 05 02:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5922(8162917) WEXITSTATUS 1 Nov 05 02:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5922(8162917) done Nov 05 02:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_10(8158584) WEXITSTATUS 0 Nov 05 02:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_10(8158584) done Nov 05 02:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5805(8162765) WEXITSTATUS 1 Nov 05 02:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5805(8162765) done Nov 05 02:20:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_4(8158578) WEXITSTATUS 0 Nov 05 02:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_4(8158578) done Nov 05 02:20:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_47(8161805) WEXITSTATUS 0 Nov 05 02:20:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_47(8161805) done Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:20:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:20:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6131(8163172) WEXITSTATUS 1 Nov 05 02:20:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6131(8163172) done Nov 05 02:20:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5830(8162797) WEXITSTATUS 1 Nov 05 02:20:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5830(8162797) done Nov 05 02:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1324(8135421) WEXITSTATUS 0 Nov 05 02:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1324(8135421) done Nov 05 02:21:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_679(7572848) Nov 05 02:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3685(8163113) WEXITSTATUS 0 Nov 05 02:21:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3685(8163113) done Nov 05 02:21:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1306(8135359) WEXITSTATUS 0 Nov 05 02:21:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1306(8135359) done Nov 05 02:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1302(8135326) WEXITSTATUS 0 Nov 05 02:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1302(8135326) done Nov 05 02:21:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6065(8163088) WEXITSTATUS 1 Nov 05 02:21:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6065(8163088) done Nov 05 02:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6052(8163075) WEXITSTATUS 1 Nov 05 02:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6052(8163075) done Nov 05 02:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159925 WEXITSTATUS 0 Nov 05 02:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159925 done Nov 05 02:21:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_44(8161802) WEXITSTATUS 0 Nov 05 02:21:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_44(8161802) done Nov 05 02:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3676(8163056) WEXITSTATUS 0 Nov 05 02:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3676(8163056) done Nov 05 02:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_2(8158576) WEXITSTATUS 0 Nov 05 02:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_2(8158576) done Nov 05 02:21:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_7(8163268) in pi_ohern on r815u03n11 Nov 05 02:21:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_8(8163269) in pi_ohern on r815u03n11 Nov 05 02:21:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_9(8163270) in pi_ohern on r815u03n12 Nov 05 02:21:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162672 in day on r913u13n03 Nov 05 02:21:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162673 in day on r914u01n02 Nov 05 02:21:43 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8163271 NodeList=(null) usec=140422 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8163271 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3701(8163272) in gpu on r808u08n01 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3702(8163273) in gpu on r808u08n01 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3703(8163274) in gpu on r4513u26n01 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3704(8163275) in gpu on r903u34n01 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1920(8163276) in day on r805u09n04 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1921(8163277) in day on r918u32n03 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1922(8163278) in day on r919u09n01 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1923(8163279) in day on r919u36n04 Nov 05 02:21:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1924(8163280) in day on r816u15n06 Nov 05 02:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_48(8161806) WEXITSTATUS 0 Nov 05 02:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_48(8161806) done Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6182(8163281) in scavenge on r908u39n04 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6183(8163282) in scavenge on r908u33n02 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6184(8163283) in scavenge on r908u33n02 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6185(8163284) in scavenge on r908u33n03 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6186(8163285) in scavenge on r908u33n03 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6187(8163286) in scavenge on r801u27n02 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6188(8163287) in scavenge on r802u02n03 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_496(7572664) in scavenge on r802u06n03 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_566(8114833) in scavenge on r816u15n12 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_500(7572668) in scavenge on r906u41n03 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1087(7572053) in scavenge on r913u24n02 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_356(7572524) in scavenge on r816u15n06 Nov 05 02:21:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_411(7572579) in scavenge on r816u35n06 Nov 05 02:21:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 02:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_54(8161812) WEXITSTATUS 0 Nov 05 02:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_54(8161812) done Nov 05 02:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159902 WEXITSTATUS 0 Nov 05 02:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159902 done Nov 05 02:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159903 WEXITSTATUS 0 Nov 05 02:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159903 done Nov 05 02:22:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_37(8161795) WEXITSTATUS 0 Nov 05 02:22:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_37(8161795) done Nov 05 02:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_49(8161807) WEXITSTATUS 0 Nov 05 02:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_49(8161807) done Nov 05 02:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2442(8127461) WEXITSTATUS 1 Nov 05 02:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2442(8127461) done Nov 05 02:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5262(8158696) WEXITSTATUS 1 Nov 05 02:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5262(8158696) done Nov 05 02:22:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1572(8150611) WEXITSTATUS 0 Nov 05 02:22:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1572(8150611) done Nov 05 02:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3702(8163273) WEXITSTATUS 0 Nov 05 02:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3702(8163273) done Nov 05 02:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1343(8135488) WEXITSTATUS 0 Nov 05 02:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1343(8135488) done Nov 05 02:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2130(8151806) WEXITSTATUS 0 Nov 05 02:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2130(8151806) done Nov 05 02:22:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159106 WEXITSTATUS 0 Nov 05 02:22:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159106 done Nov 05 02:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1376(8135671) WEXITSTATUS 0 Nov 05 02:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1376(8135671) done Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:22:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1453(8136975) WEXITSTATUS 0 Nov 05 02:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1453(8136975) done Nov 05 02:22:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_52(8161810) WEXITSTATUS 0 Nov 05 02:22:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_52(8161810) done Nov 05 02:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_115(8161893) WEXITSTATUS 0 Nov 05 02:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_115(8161893) done Nov 05 02:23:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3701(8163272) WEXITSTATUS 0 Nov 05 02:23:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3701(8163272) done Nov 05 02:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_123(8161901) WEXITSTATUS 0 Nov 05 02:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_123(8161901) done Nov 05 02:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6081(8163108) WEXITSTATUS 1 Nov 05 02:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6081(8163108) done Nov 05 02:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1288(8135278) WEXITSTATUS 0 Nov 05 02:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1288(8135278) done Nov 05 02:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2429(8152473) WEXITSTATUS 0 Nov 05 02:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2429(8152473) done Nov 05 02:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_120(8161898) WEXITSTATUS 0 Nov 05 02:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_120(8161898) done Nov 05 02:23:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3751(8142319) WEXITSTATUS 1 Nov 05 02:23:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3751(8142319) done Nov 05 02:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3703(8163274) WEXITSTATUS 0 Nov 05 02:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3703(8163274) done Nov 05 02:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_39(8161797) WEXITSTATUS 0 Nov 05 02:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_39(8161797) done Nov 05 02:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6151(8163198) WEXITSTATUS 1 Nov 05 02:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6151(8163198) done Nov 05 02:23:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_117(8161895) WEXITSTATUS 0 Nov 05 02:23:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_117(8161895) done Nov 05 02:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3697(8163227) WEXITSTATUS 0 Nov 05 02:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3697(8163227) done Nov 05 02:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154627 WEXITSTATUS 0 Nov 05 02:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154627 done Nov 05 02:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1475(8138476) WEXITSTATUS 0 Nov 05 02:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1475(8138476) done Nov 05 02:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5742(8162603) WEXITSTATUS 1 Nov 05 02:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5742(8162603) done Nov 05 02:24:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5874(8162846) WEXITSTATUS 1 Nov 05 02:24:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5874(8162846) done Nov 05 02:24:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_42(8161800) WEXITSTATUS 0 Nov 05 02:24:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_42(8161800) done Nov 05 02:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1383(8135684) WEXITSTATUS 0 Nov 05 02:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1383(8135684) done Nov 05 02:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6133(8163174) WEXITSTATUS 1 Nov 05 02:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6133(8163174) done Nov 05 02:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5824(8162786) WEXITSTATUS 1 Nov 05 02:24:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5824(8162786) done Nov 05 02:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_111(8161889) WEXITSTATUS 0 Nov 05 02:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_111(8161889) done Nov 05 02:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163246 WEXITSTATUS 130 Nov 05 02:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163246 done Nov 05 02:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1369(8135645) WEXITSTATUS 0 Nov 05 02:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1369(8135645) done Nov 05 02:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6153(8163200) WEXITSTATUS 1 Nov 05 02:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6153(8163200) done Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_10(8163288) in pi_ohern on r815u03n11 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_11(8163289) in pi_ohern on r815u03n11 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_12(8163290) in pi_ohern on r815u03n11 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_13(8163291) in pi_ohern on r815u03n11 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_14(8163292) in pi_ohern on r815u03n11 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_15(8163293) in pi_ohern on r815u03n11 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_16(8163294) in pi_ohern on r815u03n11 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_17(8163295) in pi_ohern on r815u03n12 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_18(8163296) in pi_ohern on r815u03n12 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_19(8163297) in pi_ohern on r815u03n12 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_20(8163298) in pi_ohern on r815u03n12 Nov 05 02:24:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_21(8163299) in pi_ohern on r815u03n12 Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162674 in day on r807u11n01 Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162675 in day on r909u29n03 Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_114(8161892) WEXITSTATUS 0 Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_114(8161892) done Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3705(8163300) in gpu on r903u40n01 Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3706(8163301) in gpu on r808u08n01 Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3707(8163302) in gpu on r808u08n01 Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3708(8163303) in gpu on r808u08n01 Nov 05 02:24:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154702 in day on r815u29n01 Nov 05 02:24:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1925(8163304) in day on r914u01n01 Nov 05 02:24:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1926(8163305) in day on r918u09n02 Nov 05 02:24:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1927(8163306) in day on r805u25n01 Nov 05 02:24:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1928(8163307) in day on r805u27n03 Nov 05 02:24:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1929(8163308) in day on r815u29n03 Nov 05 02:24:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1930(8163309) in day on r816u03n01 Nov 05 02:24:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1931(8163310) in day on r816u03n05 Nov 05 02:24:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3694(8163189) WEXITSTATUS 0 Nov 05 02:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3694(8163189) done Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6189(8163311) in scavenge on r801u30n04 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6190(8163312) in scavenge on r801u09n03 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6191(8163313) in scavenge on r802u32n01 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6192(8163314) in scavenge on r904u01n02 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6193(8163315) in scavenge on r807u11n01 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6194(8163316) in scavenge on r807u11n01 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6195(8163317) in scavenge on r906u41n01 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6196(8163318) in scavenge on r914u01n01 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6197(8163319) in scavenge on r805u09n03 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6198(8163320) in scavenge on r805u14n01 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6199(8163321) in scavenge on r805u25n01 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_382(8114649) in scavenge on r815u29n03 Nov 05 02:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_530(8114797) in scavenge on r816u35n06 Nov 05 02:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_348(8160641) WEXITSTATUS 0 Nov 05 02:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_348(8160641) done Nov 05 02:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_122(8161900) WEXITSTATUS 0 Nov 05 02:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_122(8161900) done Nov 05 02:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5380(8159273) WEXITSTATUS 1 Nov 05 02:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5380(8159273) done Nov 05 02:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_43(8161801) WEXITSTATUS 0 Nov 05 02:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_43(8161801) done Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:24:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:24:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_116(8161894) WEXITSTATUS 0 Nov 05 02:24:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_116(8161894) done Nov 05 02:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_347(8160640) WEXITSTATUS 0 Nov 05 02:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_347(8160640) done Nov 05 02:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_345(8160638) WEXITSTATUS 0 Nov 05 02:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_345(8160638) done Nov 05 02:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3700(8163248) WEXITSTATUS 0 Nov 05 02:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3700(8163248) done Nov 05 02:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_112(8161890) WEXITSTATUS 0 Nov 05 02:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_112(8161890) done Nov 05 02:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154607 WEXITSTATUS 0 Nov 05 02:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154607 done Nov 05 02:25:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6123(8163164) WEXITSTATUS 1 Nov 05 02:25:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6123(8163164) done Nov 05 02:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5989(8162989) WEXITSTATUS 1 Nov 05 02:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5989(8162989) done Nov 05 02:25:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5400(8159338) WEXITSTATUS 1 Nov 05 02:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5400(8159338) done Nov 05 02:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_38(8161796) WEXITSTATUS 0 Nov 05 02:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_38(8161796) done Nov 05 02:25:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_40(8161798) WEXITSTATUS 0 Nov 05 02:25:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_40(8161798) done Nov 05 02:25:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5866(8162838) WEXITSTATUS 1 Nov 05 02:25:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5866(8162838) done Nov 05 02:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5764(8162700) WEXITSTATUS 1 Nov 05 02:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5764(8162700) done Nov 05 02:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1328(8135425) WEXITSTATUS 0 Nov 05 02:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1328(8135425) done Nov 05 02:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1495(8141483) WEXITSTATUS 0 Nov 05 02:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1495(8141483) done Nov 05 02:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1285(8135268) WEXITSTATUS 0 Nov 05 02:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1285(8135268) done Nov 05 02:26:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_197(8161992) WEXITSTATUS 0 Nov 05 02:26:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_197(8161992) done Nov 05 02:26:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5815(8162777) WEXITSTATUS 1 Nov 05 02:26:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5815(8162777) done Nov 05 02:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6062(8163085) WEXITSTATUS 1 Nov 05 02:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6062(8163085) done Nov 05 02:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_36(8158668) WEXITSTATUS 0 Nov 05 02:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_36(8158668) done Nov 05 02:26:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_121(8161899) WEXITSTATUS 0 Nov 05 02:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_121(8161899) done Nov 05 02:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5594(8161557) WEXITSTATUS 1 Nov 05 02:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5594(8161557) done Nov 05 02:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_46(8161804) WEXITSTATUS 0 Nov 05 02:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_46(8161804) done Nov 05 02:26:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1372(8135667) WEXITSTATUS 0 Nov 05 02:26:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1372(8135667) done Nov 05 02:26:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_973(7573142) Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:26:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133381 WEXITSTATUS 0 Nov 05 02:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133381 done Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_22(8163322) in pi_ohern on r815u03n11 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_23(8163323) in pi_ohern on r815u03n11 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_24(8163324) in pi_ohern on r815u03n11 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_25(8163325) in pi_ohern on r815u03n11 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_26(8163326) in pi_ohern on r815u03n11 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_27(8163327) in pi_ohern on r815u03n11 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_28(8163328) in pi_ohern on r815u03n11 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_29(8163329) in pi_ohern on r815u03n11 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_30(8163330) in pi_ohern on r815u03n12 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_31(8163331) in pi_ohern on r815u03n12 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5650(8161943) WEXITSTATUS 1 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5650(8161943) done Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6172(8163258) WEXITSTATUS 1 Nov 05 02:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6172(8163258) done Nov 05 02:27:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3709(8163332) in gpu on r808u11n01 Nov 05 02:27:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3710(8163333) in gpu on r4513u26n01 Nov 05 02:27:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154703 in day on r815u35n04 Nov 05 02:27:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1932(8163334) in day on r805u04n03 Nov 05 02:27:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1933(8163335) in day on r918u01n03 Nov 05 02:27:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133396 in gpu on r4513u29n01 Nov 05 02:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_119(8161897) WEXITSTATUS 0 Nov 05 02:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_119(8161897) done Nov 05 02:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1397(8135806) WEXITSTATUS 0 Nov 05 02:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1397(8135806) done Nov 05 02:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1322(8135419) WEXITSTATUS 0 Nov 05 02:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1322(8135419) done Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6200(8163336) in scavenge on r801u04n01 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6201(8163337) in scavenge on r801u09n04 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6202(8163338) in scavenge on r815u15n11 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6203(8163339) in scavenge on r805u27n03 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6204(8163340) in scavenge on r912u28n03 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6205(8163341) in scavenge on r912u32n01 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6206(8163342) in scavenge on r918u09n04 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6207(8163343) in scavenge on r918u32n03 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6208(8163344) in scavenge on r919u05n01 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6209(8163345) in scavenge on r919u05n03 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6210(8163346) in scavenge on r919u09n02 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_533(8114800) in scavenge on r816u15n04 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_811(8115080) in scavenge on r816u35n03 Nov 05 02:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_394(7571359) in scavenge on r816u35n03 Nov 05 02:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5619(8161720) WEXITSTATUS 1 Nov 05 02:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5619(8161720) done Nov 05 02:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1368(8135644) WEXITSTATUS 0 Nov 05 02:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1368(8135644) done Nov 05 02:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_193(8161988) WEXITSTATUS 0 Nov 05 02:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_193(8161988) done Nov 05 02:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5571(8161225) WEXITSTATUS 1 Nov 05 02:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5571(8161225) done Nov 05 02:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3704(8163275) WEXITSTATUS 0 Nov 05 02:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3704(8163275) done Nov 05 02:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_45(8161803) WEXITSTATUS 0 Nov 05 02:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_45(8161803) done Nov 05 02:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_35(8161793) WEXITSTATUS 0 Nov 05 02:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_35(8161793) done Nov 05 02:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6049(8163072) WEXITSTATUS 1 Nov 05 02:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6049(8163072) done Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_346(8160639) WEXITSTATUS 0 Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2114 TRES cpu grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2114 TRES mem grp_used_tres_run_secs underflow, tried to remove 19894272 seconds when only 19648512 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2114 TRES node grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2114 TRES billing grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2113 TRES cpu grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2113 TRES mem grp_used_tres_run_secs underflow, tried to remove 19894272 seconds when only 19648512 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2113 TRES node grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2113 TRES billing grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2114 TRES cpu grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159941_346(8160639) done Nov 05 02:27:48 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2114 TRES mem grp_used_tres_run_secs underflow, tried to remove 19894272 seconds when only 19648512 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2114 TRES node grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2114 TRES billing grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2113 TRES cpu grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2113 TRES mem grp_used_tres_run_secs underflow, tried to remove 19894272 seconds when only 19648512 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2113 TRES node grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:48 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8160639: assoc 2113 TRES billing grp_used_tres_run_secs underflow, tried to remove 4857 seconds when only 4797 remained. Nov 05 02:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1422(8135955) WEXITSTATUS 0 Nov 05 02:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1422(8135955) done Nov 05 02:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_198(8161993) WEXITSTATUS 0 Nov 05 02:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_198(8161993) done Nov 05 02:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5713(8162426) WEXITSTATUS 1 Nov 05 02:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5713(8162426) done Nov 05 02:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6206(8163342) WEXITSTATUS 1 Nov 05 02:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6206(8163342) done Nov 05 02:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5610(8161711) WEXITSTATUS 1 Nov 05 02:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5610(8161711) done Nov 05 02:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5255(8158679) WEXITSTATUS 1 Nov 05 02:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5255(8158679) done Nov 05 02:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_118(8161896) WEXITSTATUS 0 Nov 05 02:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_118(8161896) done Nov 05 02:28:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1565(8149123) WEXITSTATUS 0 Nov 05 02:28:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1565(8149123) done Nov 05 02:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_65(8161823) WEXITSTATUS 0 Nov 05 02:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_65(8161823) done Nov 05 02:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_41(8161799) WEXITSTATUS 0 Nov 05 02:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_41(8161799) done Nov 05 02:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_3(8158577) WEXITSTATUS 0 Nov 05 02:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_3(8158577) done Nov 05 02:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5804(8162763) WEXITSTATUS 1 Nov 05 02:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5804(8162763) done Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:28:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:28:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5081(8158393) WEXITSTATUS 1 Nov 05 02:28:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5081(8158393) done Nov 05 02:28:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5910(8162902) WEXITSTATUS 1 Nov 05 02:28:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5910(8162902) done Nov 05 02:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_36(8161794) WEXITSTATUS 0 Nov 05 02:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_36(8161794) done Nov 05 02:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1318(8135395) WEXITSTATUS 0 Nov 05 02:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1318(8135395) done Nov 05 02:29:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5505(8160508) WEXITSTATUS 1 Nov 05 02:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5505(8160508) done Nov 05 02:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5813(8162773) WEXITSTATUS 1 Nov 05 02:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5813(8162773) done Nov 05 02:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5927(8162922) WEXITSTATUS 1 Nov 05 02:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5927(8162922) done Nov 05 02:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5958(8162954) WEXITSTATUS 1 Nov 05 02:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5958(8162954) done Nov 05 02:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_113(8161891) WEXITSTATUS 0 Nov 05 02:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_113(8161891) done Nov 05 02:29:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163271 WEXITSTATUS 0 Nov 05 02:29:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163271 done Nov 05 02:29:43 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8163347 NodeList=(null) usec=26936 Nov 05 02:29:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8163347 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 02:29:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8162105 Nov 05 02:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_203(8161998) WEXITSTATUS 0 Nov 05 02:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_203(8161998) done Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_32(8163348) in pi_ohern on r815u03n11 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_33(8163349) in pi_ohern on r815u03n11 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_34(8163350) in pi_ohern on r815u03n11 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_35(8163351) in pi_ohern on r815u03n11 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_36(8163352) in pi_ohern on r815u03n11 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_37(8163353) in pi_ohern on r815u03n11 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_38(8163354) in pi_ohern on r815u03n12 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_39(8163355) in pi_ohern on r815u03n12 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_40(8163356) in pi_ohern on r815u03n12 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_41(8163357) in pi_ohern on r815u03n12 Nov 05 02:29:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_42(8163358) in pi_ohern on r903u24n01 Nov 05 02:29:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163224 in day on r805u04n04,r919u32n02 Nov 05 02:29:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3711(8163359) in gpu on r903u34n01 Nov 05 02:29:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1934(8163360) in day on r805u14n02 Nov 05 02:29:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1935(8163361) in day on r805u14n03 Nov 05 02:29:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1936(8163362) in day on r816u15n07 Nov 05 02:29:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6211(8163363) in scavenge on r908u33n01 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6212(8163364) in scavenge on r907u01n03 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6213(8163365) in scavenge on r907u09n02 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6214(8163366) in scavenge on r908u35n01 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6215(8163367) in scavenge on r903u01n01 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6216(8163368) in scavenge on r903u07n01 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6217(8163369) in scavenge on r815u23n12 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6218(8163370) in scavenge on r816u23n11 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6219(8163371) in scavenge on r904u07n04 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6220(8163372) in scavenge on r904u04n03 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6221(8163373) in scavenge on r913u01n04 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6222(8163374) in scavenge on r805u14n03 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6223(8163375) in scavenge on r805u23n04 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6224(8163376) in scavenge on r918u09n04 Nov 05 02:29:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_19(8114285) in scavenge on r816u15n03 Nov 05 02:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_194(8161989) WEXITSTATUS 0 Nov 05 02:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_194(8161989) done Nov 05 02:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5902(8162894) WEXITSTATUS 1 Nov 05 02:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5902(8162894) done Nov 05 02:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_199(8161994) WEXITSTATUS 0 Nov 05 02:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_199(8161994) done Nov 05 02:30:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5342(8159020) WEXITSTATUS 1 Nov 05 02:30:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5342(8159020) done Nov 05 02:30:02 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 02:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5896(8162872) WEXITSTATUS 1 Nov 05 02:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5896(8162872) done Nov 05 02:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154596 WEXITSTATUS 0 Nov 05 02:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154596 done Nov 05 02:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3711(8163359) WEXITSTATUS 0 Nov 05 02:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3711(8163359) done Nov 05 02:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1337(8135475) WEXITSTATUS 0 Nov 05 02:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1337(8135475) done Nov 05 02:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159103 WEXITSTATUS 0 Nov 05 02:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159103 done Nov 05 02:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_263(8162068) WEXITSTATUS 0 Nov 05 02:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_263(8162068) done Nov 05 02:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159104 WEXITSTATUS 0 Nov 05 02:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159104 done Nov 05 02:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1364(8135636) WEXITSTATUS 0 Nov 05 02:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1364(8135636) done Nov 05 02:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1384(8123615) WEXITSTATUS 1 Nov 05 02:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1384(8123615) done Nov 05 02:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1313(8135385) WEXITSTATUS 0 Nov 05 02:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1313(8135385) done Nov 05 02:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154655 WEXITSTATUS 0 Nov 05 02:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154655 done Nov 05 02:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4497(8155880) WEXITSTATUS 1 Nov 05 02:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4497(8155880) done Nov 05 02:30:45 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2948(8128632) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:30:46 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2948(8128632) Nov 05 02:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6190(8163312) WEXITSTATUS 1 Nov 05 02:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6190(8163312) done Nov 05 02:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3627(8161836) WEXITSTATUS 0 Nov 05 02:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3627(8161836) done Nov 05 02:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_204(8161999) WEXITSTATUS 0 Nov 05 02:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_204(8161999) done Nov 05 02:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1395(8135804) WEXITSTATUS 0 Nov 05 02:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1395(8135804) done Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:31:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:31:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1392(8135801) WEXITSTATUS 0 Nov 05 02:31:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1392(8135801) done Nov 05 02:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_195(8161990) WEXITSTATUS 0 Nov 05 02:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_195(8161990) done Nov 05 02:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3707(8163302) WEXITSTATUS 0 Nov 05 02:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3707(8163302) done Nov 05 02:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5976(8162976) WEXITSTATUS 1 Nov 05 02:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5976(8162976) done Nov 05 02:31:16 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5727(8162560) WEXITSTATUS 1 Nov 05 02:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5727(8162560) done Nov 05 02:31:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_327(8162141) WEXITSTATUS 0 Nov 05 02:31:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_327(8162141) done Nov 05 02:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_200(8161995) WEXITSTATUS 0 Nov 05 02:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_200(8161995) done Nov 05 02:31:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1629(8155098) WEXITSTATUS 0 Nov 05 02:31:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1629(8155098) done Nov 05 02:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_55(8161813) WEXITSTATUS 0 Nov 05 02:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_55(8161813) done Nov 05 02:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_321(8162135) WEXITSTATUS 0 Nov 05 02:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_321(8162135) done Nov 05 02:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6070(8163097) WEXITSTATUS 1 Nov 05 02:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6070(8163097) done Nov 05 02:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_128(8161906) WEXITSTATUS 0 Nov 05 02:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_128(8161906) done Nov 05 02:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_1(8161740) WEXITSTATUS 0 Nov 05 02:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_1(8161740) done Nov 05 02:31:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_202(8161997) WEXITSTATUS 0 Nov 05 02:31:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_202(8161997) done Nov 05 02:31:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_261(8162066) WEXITSTATUS 0 Nov 05 02:31:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_261(8162066) done Nov 05 02:31:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1319(8135405) WEXITSTATUS 0 Nov 05 02:31:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1319(8135405) done Nov 05 02:31:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1271(8135220) WEXITSTATUS 0 Nov 05 02:31:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1271(8135220) done Nov 05 02:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1292(8135298) WEXITSTATUS 0 Nov 05 02:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1292(8135298) done Nov 05 02:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_324(8162138) WEXITSTATUS 0 Nov 05 02:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_324(8162138) done Nov 05 02:32:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1325(8135422) WEXITSTATUS 0 Nov 05 02:32:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1325(8135422) done Nov 05 02:32:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5977(8162977) WEXITSTATUS 1 Nov 05 02:32:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5977(8162977) done Nov 05 02:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5274(8158708) WEXITSTATUS 1 Nov 05 02:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5274(8158708) done Nov 05 02:32:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_2(8161741) WEXITSTATUS 0 Nov 05 02:32:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_2(8161741) done Nov 05 02:32:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_201(8161996) WEXITSTATUS 0 Nov 05 02:32:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_201(8161996) done Nov 05 02:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162174 WEXITSTATUS 0 Nov 05 02:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162174 done Nov 05 02:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3686(8163133) WEXITSTATUS 0 Nov 05 02:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3686(8163133) done Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_43(8163377) in pi_ohern on r815u03n11 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_44(8163378) in pi_ohern on r815u03n11 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_45(8163379) in pi_ohern on r815u03n11 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_46(8163380) in pi_ohern on r815u03n11 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_47(8163381) in pi_ohern on r815u03n11 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_48(8163382) in pi_ohern on r815u03n12 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_49(8163383) in pi_ohern on r815u03n12 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_50(8163384) in pi_ohern on r815u03n12 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_51(8163385) in pi_ohern on r815u03n12 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_52(8163386) in pi_ohern on r815u03n12 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_53(8163387) in pi_ohern on r815u03n12 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_54(8163388) in pi_ohern on r815u03n12 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_55(8163389) in pi_ohern on r903u26n01 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_56(8163390) in pi_ohern on r903u26n01 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_57(8163391) in pi_ohern on r911u32n01 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_58(8163392) in pi_ohern on r913u32n01 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 05 02:32:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n04 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_333(8162147) WEXITSTATUS 0 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_333(8162147) done Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3712(8163393) in gpu on r903u34n01 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3713(8163394) in gpu on r903u34n01 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3714(8163395) in gpu on r808u08n01 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3715(8163396) in gpu on r4513u23n01 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154699 in day on r806u02n01 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154705 in day on r807u03n03 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1937(8163397) in day on r807u03n02 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1938(8163398) in day on r914u01n02 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1939(8163399) in day on r805u02n01 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1940(8163400) in day on r805u23n02 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1941(8163401) in day on r919u09n03 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1942(8163402) in day on r919u32n02 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1943(8163403) in day on r912u36n02 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1944(8163404) in day on r913u05n01 Nov 05 02:32:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1945(8163405) in day on r913u09n04 Nov 05 02:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162173 WEXITSTATUS 0 Nov 05 02:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162173 done Nov 05 02:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_262(8162067) WEXITSTATUS 0 Nov 05 02:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_262(8162067) done Nov 05 02:32:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5899(8162875) WEXITSTATUS 1 Nov 05 02:32:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5899(8162875) done Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6225(8163406) in scavenge on r801u09n03 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6226(8163407) in scavenge on r903u01n01 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6227(8163408) in scavenge on r912u05n04 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6228(8163409) in scavenge on r802u09n03 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6229(8163410) in scavenge on r802u11n01 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6230(8163411) in scavenge on r816u09n10 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6231(8163412) in scavenge on r816u35n11 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6232(8163413) in scavenge on r806u02n01 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6233(8163414) in scavenge on r805u02n01 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6234(8163415) in scavenge on r805u11n03 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6235(8163416) in scavenge on r807u03n02 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6236(8163417) in scavenge on r807u03n03 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6237(8163418) in scavenge on r807u03n03 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_503(7572671) in scavenge on r913u05n01 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_513(7572681) in scavenge on r913u09n04 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_662(7571627) in scavenge on r913u09n04 Nov 05 02:32:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_255(8114521) in scavenge on r816u03n02 Nov 05 02:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_329(8162143) WEXITSTATUS 0 Nov 05 02:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_329(8162143) done Nov 05 02:32:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162171 WEXITSTATUS 0 Nov 05 02:32:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162171 done Nov 05 02:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1404(8135937) WEXITSTATUS 0 Nov 05 02:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1404(8135937) done Nov 05 02:32:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8135428 Nov 05 02:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5864(8162836) WEXITSTATUS 1 Nov 05 02:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5864(8162836) done Nov 05 02:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3714(8163395) WEXITSTATUS 0 Nov 05 02:32:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3714(8163395) done Nov 05 02:32:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 02:32:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 02:32:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6073(8163100) WEXITSTATUS 1 Nov 05 02:33:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6073(8163100) done Nov 05 02:33:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_322(8162136) WEXITSTATUS 0 Nov 05 02:33:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_322(8162136) done Nov 05 02:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_264(8162069) WEXITSTATUS 0 Nov 05 02:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_264(8162069) done Nov 05 02:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162172 WEXITSTATUS 0 Nov 05 02:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162172 done Nov 05 02:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_45(8158715) WEXITSTATUS 0 Nov 05 02:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_45(8158715) done Nov 05 02:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_260(8162065) WEXITSTATUS 0 Nov 05 02:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_260(8162065) done Nov 05 02:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1317(8135394) WEXITSTATUS 0 Nov 05 02:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1317(8135394) done Nov 05 02:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_320(8162134) WEXITSTATUS 0 Nov 05 02:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_320(8162134) done Nov 05 02:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_326(8162140) WEXITSTATUS 0 Nov 05 02:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_326(8162140) done Nov 05 02:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5033(8158329) WEXITSTATUS 1 Nov 05 02:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5033(8158329) done Nov 05 02:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5786(8162741) WEXITSTATUS 1 Nov 05 02:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5786(8162741) done Nov 05 02:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_56(8161814) WEXITSTATUS 0 Nov 05 02:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_56(8161814) done Nov 05 02:33:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8142529 NodeList=r815u03n04 #CPUs=48 Partition=day Nov 05 02:33:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_23(8161781) WEXITSTATUS 0 Nov 05 02:33:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_23(8161781) done Nov 05 02:33:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162170 WEXITSTATUS 0 Nov 05 02:33:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162170 done Nov 05 02:33:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_57(8161815) WEXITSTATUS 0 Nov 05 02:33:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_57(8161815) done Nov 05 02:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1381(8135682) WEXITSTATUS 0 Nov 05 02:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1381(8135682) done Nov 05 02:33:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5827(8162789) WEXITSTATUS 1 Nov 05 02:33:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5827(8162789) done Nov 05 02:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_125(8161903) WEXITSTATUS 0 Nov 05 02:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_125(8161903) done Nov 05 02:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5941(8162936) WEXITSTATUS 1 Nov 05 02:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5941(8162936) done Nov 05 02:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_328(8162142) WEXITSTATUS 0 Nov 05 02:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_328(8162142) done Nov 05 02:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162180 WEXITSTATUS 0 Nov 05 02:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162180 done Nov 05 02:34:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3713(8163394) WEXITSTATUS 0 Nov 05 02:34:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3713(8163394) done Nov 05 02:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_59(8161817) WEXITSTATUS 0 Nov 05 02:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_59(8161817) done Nov 05 02:34:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_323(8162137) WEXITSTATUS 0 Nov 05 02:34:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_323(8162137) done Nov 05 02:34:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162623 WEXITSTATUS 0 Nov 05 02:34:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162623 done Nov 05 02:34:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1508(8143364) WEXITSTATUS 0 Nov 05 02:34:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1508(8143364) done Nov 05 02:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3712(8163393) WEXITSTATUS 0 Nov 05 02:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3712(8163393) done Nov 05 02:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_146(8161924) WEXITSTATUS 0 Nov 05 02:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_146(8161924) done Nov 05 02:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5790(8162745) WEXITSTATUS 1 Nov 05 02:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5790(8162745) done Nov 05 02:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5385(8159323) WEXITSTATUS 1 Nov 05 02:34:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5385(8159323) done Nov 05 02:34:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_384(8162229) WEXITSTATUS 0 Nov 05 02:34:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_384(8162229) done Nov 05 02:34:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162177 WEXITSTATUS 0 Nov 05 02:34:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162177 done Nov 05 02:34:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_391(8162236) WEXITSTATUS 0 Nov 05 02:34:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_391(8162236) done Nov 05 02:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 02:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 02:34:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:35:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_61(8161819) WEXITSTATUS 0 Nov 05 02:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_61(8161819) done Nov 05 02:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_63(8161821) WEXITSTATUS 0 Nov 05 02:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_63(8161821) done Nov 05 02:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5384(8159322) WEXITSTATUS 1 Nov 05 02:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5384(8159322) done Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_59(8163419) in pi_ohern on r917u13n04 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_60(8163420) in pi_ohern on r815u03n11 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_61(8163421) in pi_ohern on r815u03n11 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_62(8163422) in pi_ohern on r815u03n11 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_63(8163423) in pi_ohern on r815u03n11 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_64(8163424) in pi_ohern on r815u03n11 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_65(8163425) in pi_ohern on r815u03n11 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_66(8163426) in pi_ohern on r815u03n11 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_67(8163427) in pi_ohern on r815u03n12 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_68(8163428) in pi_ohern on r815u03n12 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_69(8163429) in pi_ohern on r815u03n12 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_70(8163430) in pi_ohern on r815u03n12 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_71(8163431) in pi_ohern on r815u03n12 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_72(8163432) in pi_ohern on r903u12n01 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_73(8163433) in pi_ohern on r903u12n01 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_74(8163434) in pi_ohern on r903u12n01 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_75(8163435) in pi_ohern on r903u12n01 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_76(8163436) in pi_ohern on r911u32n01 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_77(8163437) in pi_ohern on r913u32n01 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_78(8163438) in pi_ohern on r913u35n01 Nov 05 02:35:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3716(8163439) in gpu on r808u08n01 Nov 05 02:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3717(8163440) in gpu on r903u34n01 Nov 05 02:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3718(8163441) in gpu on r903u34n01 Nov 05 02:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1946(8163442) in day on r805u06n02 Nov 05 02:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1947(8163443) in day on r805u27n02 Nov 05 02:35:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6238(8163444) in scavenge on r911u24n03 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6239(8163445) in scavenge on r802u02n02 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6240(8163446) in scavenge on r802u04n04 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6241(8163447) in scavenge on r815u03n09 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6242(8163448) in scavenge on r815u23n07 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6243(8163449) in scavenge on r816u23n10 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6244(8163450) in scavenge on r816u23n12 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6245(8163451) in scavenge on r816u29n11 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6246(8163452) in scavenge on r912u32n01 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_514(7572682) in scavenge on r805u06n02 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_571(7572740) in scavenge on r805u06n02 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_643(7572812) in scavenge on r806u06n02 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_656(7572825) in scavenge on r912u32n01 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_314(8114581) in scavenge on r815u35n02 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_912(8116386) in scavenge on r816u03n04 Nov 05 02:35:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1177(7573346) in scavenge on r816u35n01 Nov 05 02:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5352(8159086) WEXITSTATUS 1 Nov 05 02:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5352(8159086) done Nov 05 02:35:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_394(8162239) WEXITSTATUS 0 Nov 05 02:35:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_394(8162239) done Nov 05 02:35:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5806(8162766) WEXITSTATUS 1 Nov 05 02:35:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5806(8162766) done Nov 05 02:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_196(8161991) WEXITSTATUS 0 Nov 05 02:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_196(8161991) done Nov 05 02:35:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3716(8163439) WEXITSTATUS 0 Nov 05 02:35:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3716(8163439) done Nov 05 02:35:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162178 WEXITSTATUS 0 Nov 05 02:35:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162178 done Nov 05 02:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_69(8161827) WEXITSTATUS 0 Nov 05 02:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_69(8161827) done Nov 05 02:35:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_392(8162237) WEXITSTATUS 0 Nov 05 02:35:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_392(8162237) done Nov 05 02:35:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_259(8162064) WEXITSTATUS 0 Nov 05 02:35:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_259(8162064) done Nov 05 02:36:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_31(8161789) WEXITSTATUS 0 Nov 05 02:36:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_31(8161789) done Nov 05 02:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_388(8162233) WEXITSTATUS 0 Nov 05 02:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_388(8162233) done Nov 05 02:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_129(8161907) WEXITSTATUS 0 Nov 05 02:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_129(8161907) done Nov 05 02:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_387(8162232) WEXITSTATUS 0 Nov 05 02:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_387(8162232) done Nov 05 02:36:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154642 WEXITSTATUS 0 Nov 05 02:36:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154642 done Nov 05 02:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_379(8162224) WEXITSTATUS 0 Nov 05 02:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_379(8162224) done Nov 05 02:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_67(8161825) WEXITSTATUS 0 Nov 05 02:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_67(8161825) done Nov 05 02:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_26(8161784) WEXITSTATUS 0 Nov 05 02:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_26(8161784) done Nov 05 02:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3705(8163300) WEXITSTATUS 0 Nov 05 02:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3705(8163300) done Nov 05 02:36:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_58(8161816) WEXITSTATUS 0 Nov 05 02:36:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_58(8161816) done Nov 05 02:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5429(8159553) WEXITSTATUS 1 Nov 05 02:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5429(8159553) done Nov 05 02:36:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_386(8162231) WEXITSTATUS 0 Nov 05 02:36:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_386(8162231) done Nov 05 02:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_126(8161904) WEXITSTATUS 0 Nov 05 02:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_126(8161904) done Nov 05 02:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_380(8162225) WEXITSTATUS 0 Nov 05 02:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_380(8162225) done Nov 05 02:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_60(8161818) WEXITSTATUS 0 Nov 05 02:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_60(8161818) done Nov 05 02:36:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1175(7573344) Nov 05 02:36:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2257(8126165) Nov 05 02:36:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:36:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:36:46 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_132(8161910) WEXITSTATUS 0 Nov 05 02:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_132(8161910) done Nov 05 02:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_439(8162295) WEXITSTATUS 0 Nov 05 02:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_439(8162295) done Nov 05 02:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_334(8162148) WEXITSTATUS 0 Nov 05 02:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_334(8162148) done Nov 05 02:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_64(8161822) WEXITSTATUS 0 Nov 05 02:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_64(8161822) done Nov 05 02:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_332(8162146) WEXITSTATUS 0 Nov 05 02:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_332(8162146) done Nov 05 02:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_393(8162238) WEXITSTATUS 0 Nov 05 02:36:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_393(8162238) done Nov 05 02:36:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_434(8162290) WEXITSTATUS 0 Nov 05 02:36:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_434(8162290) done Nov 05 02:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6177(8163263) WEXITSTATUS 1 Nov 05 02:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6177(8163263) done Nov 05 02:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5268(8158702) WEXITSTATUS 1 Nov 05 02:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5268(8158702) done Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:37:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:37:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6224(8163376) WEXITSTATUS 1 Nov 05 02:37:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6224(8163376) done Nov 05 02:37:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_389(8162234) WEXITSTATUS 0 Nov 05 02:37:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_389(8162234) done Nov 05 02:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4846(8157854) WEXITSTATUS 1 Nov 05 02:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4846(8157854) done Nov 05 02:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8142529 WEXITSTATUS 0 Nov 05 02:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8142529 done Nov 05 02:37:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_66(8161824) WEXITSTATUS 0 Nov 05 02:37:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_66(8161824) done Nov 05 02:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1385(8135686) WEXITSTATUS 0 Nov 05 02:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1385(8135686) done Nov 05 02:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1308(8135366) WEXITSTATUS 0 Nov 05 02:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1308(8135366) done Nov 05 02:37:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162179 WEXITSTATUS 0 Nov 05 02:37:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162179 done Nov 05 02:37:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5478(8159901) WEXITSTATUS 1 Nov 05 02:37:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5478(8159901) done Nov 05 02:37:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1409(8135942) WEXITSTATUS 0 Nov 05 02:37:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1409(8135942) done Nov 05 02:37:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_74(8161832) WEXITSTATUS 0 Nov 05 02:37:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_74(8161832) done Nov 05 02:37:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8125757 NodeList=r815u03n04,r815u15n06 #CPUs=64 Partition=day Nov 05 02:37:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162183 WEXITSTATUS 0 Nov 05 02:37:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162183 done Nov 05 02:37:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_72(8161830) WEXITSTATUS 0 Nov 05 02:37:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_72(8161830) done Nov 05 02:37:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_330(8162144) WEXITSTATUS 0 Nov 05 02:37:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_330(8162144) done Nov 05 02:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5724(8162526) WEXITSTATUS 1 Nov 05 02:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5724(8162526) done Nov 05 02:37:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5842(8162813) WEXITSTATUS 1 Nov 05 02:37:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5842(8162813) done Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_79(8163453) in pi_ohern on r917u13n04 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_80(8163454) in pi_ohern on r917u13n04 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_81(8163455) in pi_ohern on r815u03n11 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_82(8163456) in pi_ohern on r815u03n11 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_83(8163457) in pi_ohern on r815u03n11 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_84(8163458) in pi_ohern on r815u03n11 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_85(8163459) in pi_ohern on r815u03n11 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_86(8163460) in pi_ohern on r815u03n11 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_87(8163461) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_88(8163462) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_89(8163463) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_90(8163464) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_91(8163465) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_92(8163466) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_93(8163467) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_94(8163468) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_95(8163469) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_96(8163470) in pi_ohern on r815u03n12 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_97(8163471) in pi_ohern on r807u24n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_98(8163472) in pi_ohern on r911u32n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_99(8163473) in pi_ohern on r911u32n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_100(8163474) in pi_ohern on r903u12n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_101(8163475) in pi_ohern on r903u12n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_102(8163476) in pi_ohern on r903u24n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_103(8163477) in pi_ohern on r903u24n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_104(8163478) in pi_ohern on r903u26n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_105(8163479) in pi_ohern on r903u26n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_106(8163480) in pi_ohern on r903u26n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_107(8163481) in pi_ohern on r903u26n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3719(8163482) in gpu on r808u08n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3720(8163483) in gpu on r903u40n01 Nov 05 02:37:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154706 in day on r913u28n03 Nov 05 02:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162181 WEXITSTATUS 0 Nov 05 02:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162181 done Nov 05 02:38:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1948(8163484) in day on r805u25n04 Nov 05 02:38:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1949(8163485) in day on r806u09n02 Nov 05 02:38:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1950(8163486) in day on r815u23n02 Nov 05 02:38:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1951(8163487) in day on r815u23n02 Nov 05 02:38:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 02:38:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_141(8161919) WEXITSTATUS 0 Nov 05 02:38:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_141(8161919) done Nov 05 02:38:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_385(8162230) WEXITSTATUS 0 Nov 05 02:38:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_385(8162230) done Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6247(8163488) in scavenge on r801u30n02 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6248(8163489) in scavenge on r801u06n01 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6249(8163490) in scavenge on r802u11n02 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6250(8163491) in scavenge on r802u11n03 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6251(8163492) in scavenge on r802u30n01 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6252(8163493) in scavenge on r816u23n07 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6253(8163494) in scavenge on r816u35n10 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6254(8163495) in scavenge on r808u11n01 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6255(8163496) in scavenge on r806u06n01 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6256(8163497) in scavenge on r806u06n01 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6257(8163498) in scavenge on r806u09n02 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_567(8114834) in scavenge on r805u23n03 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_516(7572684) in scavenge on r815u23n02 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_731(7572900) in scavenge on r815u23n02 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_816(8115085) in scavenge on r816u29n06 Nov 05 02:38:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_107(8115577) in scavenge on r816u35n01 Nov 05 02:38:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162661 WEXITSTATUS 0 Nov 05 02:38:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162661 done Nov 05 02:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6213(8163365) WEXITSTATUS 1 Nov 05 02:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6213(8163365) done Nov 05 02:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_208(8162003) WEXITSTATUS 0 Nov 05 02:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_208(8162003) done Nov 05 02:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_130(8161908) WEXITSTATUS 0 Nov 05 02:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_130(8161908) done Nov 05 02:38:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6198(8163320) WEXITSTATUS 1 Nov 05 02:38:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6198(8163320) done Nov 05 02:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1465(8138360) WEXITSTATUS 0 Nov 05 02:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1465(8138360) done Nov 05 02:38:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_438(8162294) WEXITSTATUS 0 Nov 05 02:38:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_438(8162294) done Nov 05 02:38:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_383(8162228) WEXITSTATUS 0 Nov 05 02:38:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_383(8162228) done Nov 05 02:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6250(8163491) WEXITSTATUS 1 Nov 05 02:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6250(8163491) done Nov 05 02:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_144(8161922) WEXITSTATUS 0 Nov 05 02:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_144(8161922) done Nov 05 02:38:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6037(8163053) WEXITSTATUS 1 Nov 05 02:38:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6037(8163053) done Nov 05 02:38:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162676 NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 02:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162182 WEXITSTATUS 0 Nov 05 02:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162182 done Nov 05 02:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_289(8162094) WEXITSTATUS 0 Nov 05 02:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_289(8162094) done Nov 05 02:38:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_75(8161833) WEXITSTATUS 0 Nov 05 02:38:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_75(8161833) done Nov 05 02:38:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1379(8135680) WEXITSTATUS 0 Nov 05 02:38:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1379(8135680) done Nov 05 02:38:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_140(8161918) WEXITSTATUS 0 Nov 05 02:38:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_140(8161918) done Nov 05 02:38:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_154(8161932) WEXITSTATUS 0 Nov 05 02:38:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_154(8161932) done Nov 05 02:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_62(8161820) WEXITSTATUS 0 Nov 05 02:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_62(8161820) done Nov 05 02:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5432(8159556) WEXITSTATUS 1 Nov 05 02:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5432(8159556) done Nov 05 02:38:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1589(8151954) WEXITSTATUS 0 Nov 05 02:38:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1589(8151954) done Nov 05 02:38:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_390(8162235) WEXITSTATUS 0 Nov 05 02:38:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_390(8162235) done Nov 05 02:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4510(8155899) WEXITSTATUS 1 Nov 05 02:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4510(8155899) done Nov 05 02:39:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_133(8161911) WEXITSTATUS 0 Nov 05 02:39:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_133(8161911) done Nov 05 02:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_80(8161858) WEXITSTATUS 0 Nov 05 02:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_80(8161858) done Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:39:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_378(8162223) WEXITSTATUS 0 Nov 05 02:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_378(8162223) done Nov 05 02:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6237(8163418) WEXITSTATUS 1 Nov 05 02:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6237(8163418) done Nov 05 02:39:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6254(8163495) WEXITSTATUS 1 Nov 05 02:39:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6254(8163495) done Nov 05 02:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_28(8161786) WEXITSTATUS 0 Nov 05 02:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_28(8161786) done Nov 05 02:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1472(8138473) WEXITSTATUS 0 Nov 05 02:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1472(8138473) done Nov 05 02:39:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162176 WEXITSTATUS 0 Nov 05 02:39:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162176 done Nov 05 02:39:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6027(8163043) WEXITSTATUS 1 Nov 05 02:39:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6027(8163043) done Nov 05 02:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_3(8158752) WEXITSTATUS 0 Nov 05 02:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_3(8158752) done Nov 05 02:39:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162175 WEXITSTATUS 0 Nov 05 02:39:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162175 done Nov 05 02:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_435(8162291) WEXITSTATUS 0 Nov 05 02:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_435(8162291) done Nov 05 02:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_124(8161902) WEXITSTATUS 0 Nov 05 02:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_124(8161902) done Nov 05 02:39:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_150(8161928) WEXITSTATUS 0 Nov 05 02:39:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_150(8161928) done Nov 05 02:39:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5210(8158597) WEXITSTATUS 1 Nov 05 02:39:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5210(8158597) done Nov 05 02:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4918(8158081) WEXITSTATUS 1 Nov 05 02:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4918(8158081) done Nov 05 02:39:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162185 WEXITSTATUS 0 Nov 05 02:39:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162185 done Nov 05 02:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_207(8162002) WEXITSTATUS 0 Nov 05 02:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_207(8162002) done Nov 05 02:39:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2264(8126174) Nov 05 02:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1375(8135670) WEXITSTATUS 0 Nov 05 02:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1375(8135670) done Nov 05 02:39:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_211(8162006) WEXITSTATUS 0 Nov 05 02:39:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_211(8162006) done Nov 05 02:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154631 WEXITSTATUS 0 Nov 05 02:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154631 done Nov 05 02:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_145(8161923) WEXITSTATUS 0 Nov 05 02:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_145(8161923) done Nov 05 02:39:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3719(8163482) WEXITSTATUS 0 Nov 05 02:39:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3719(8163482) done Nov 05 02:40:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2619(8127876) WEXITSTATUS 1 Nov 05 02:40:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2619(8127876) done Nov 05 02:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_27(8161785) WEXITSTATUS 0 Nov 05 02:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_27(8161785) done Nov 05 02:40:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_325(8162139) WEXITSTATUS 0 Nov 05 02:40:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_325(8162139) done Nov 05 02:40:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_436(8162292) WEXITSTATUS 0 Nov 05 02:40:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_436(8162292) done Nov 05 02:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_25(8161783) WEXITSTATUS 0 Nov 05 02:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_25(8161783) done Nov 05 02:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_220(8162015) WEXITSTATUS 0 Nov 05 02:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_220(8162015) done Nov 05 02:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_68(8161826) WEXITSTATUS 0 Nov 05 02:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_68(8161826) done Nov 05 02:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5145(8158492) WEXITSTATUS 1 Nov 05 02:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5145(8158492) done Nov 05 02:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6251(8163492) WEXITSTATUS 1 Nov 05 02:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6251(8163492) done Nov 05 02:40:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163499 InitPrio=21238 usec=3245 Nov 05 02:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_382(8162227) WEXITSTATUS 0 Nov 05 02:40:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_382(8162227) done Nov 05 02:40:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5988(8162988) WEXITSTATUS 1 Nov 05 02:40:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5988(8162988) done Nov 05 02:40:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_226(8162021) WEXITSTATUS 0 Nov 05 02:40:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_226(8162021) done Nov 05 02:40:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_134(8161912) WEXITSTATUS 0 Nov 05 02:40:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_134(8161912) done Nov 05 02:40:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8163499 NodeList=r815u03n06,r815u15n05 #CPUs=72 Partition=day Nov 05 02:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_437(8162293) WEXITSTATUS 0 Nov 05 02:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_437(8162293) done Nov 05 02:40:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_221(8162016) WEXITSTATUS 0 Nov 05 02:40:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_221(8162016) done Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_108(8163500) in pi_ohern on r801u32n02 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_109(8163501) in pi_ohern on r917u13n04 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_110(8163502) in pi_ohern on r917u13n04 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_111(8163503) in pi_ohern on r917u13n04 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_112(8163504) in pi_ohern on r815u03n11 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_113(8163505) in pi_ohern on r815u03n11 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_114(8163506) in pi_ohern on r815u03n11 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_115(8163507) in pi_ohern on r815u03n11 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_116(8163508) in pi_ohern on r815u03n11 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_117(8163509) in pi_ohern on r815u03n12 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_118(8163510) in pi_ohern on r815u03n12 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_119(8163511) in pi_ohern on r815u03n12 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_120(8163512) in pi_ohern on r815u03n12 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_121(8163513) in pi_ohern on r815u03n12 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_122(8163514) in pi_ohern on r807u24n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_123(8163515) in pi_ohern on r911u32n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_124(8163516) in pi_ohern on r911u32n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_125(8163517) in pi_ohern on r911u32n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_126(8163518) in pi_ohern on r911u32n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_127(8163519) in pi_ohern on r911u32n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_128(8163520) in pi_ohern on r913u32n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_129(8163521) in pi_ohern on r913u35n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_130(8163522) in pi_ohern on r913u35n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_131(8163523) in pi_ohern on r913u35n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_132(8163524) in pi_ohern on r913u35n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_133(8163525) in pi_ohern on r903u12n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_134(8163526) in pi_ohern on r903u12n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_135(8163527) in pi_ohern on r903u12n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_136(8163528) in pi_ohern on r903u12n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_137(8163529) in pi_ohern on r903u24n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_138(8163530) in pi_ohern on r903u26n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_139(8163531) in pi_ohern on r903u26n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_140(8163532) in pi_ohern on r903u26n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_141(8163533) in pi_ohern on r903u26n01 Nov 05 02:40:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3721(8163534) in gpu on r808u08n01 Nov 05 02:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6220(8163372) WEXITSTATUS 1 Nov 05 02:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6220(8163372) done Nov 05 02:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_19(8114285) WEXITSTATUS 0 Nov 05 02:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_19(8114285) done Nov 05 02:40:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1952(8163535) in day on r807u08n01 Nov 05 02:40:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1953(8163536) in day on r815u03n06 Nov 05 02:40:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1954(8163537) in day on r815u15n05 Nov 05 02:40:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1955(8163538) in day on r815u15n05 Nov 05 02:40:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1956(8163539) in day on r815u15n05 Nov 05 02:40:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:40:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_24(8161782) WEXITSTATUS 0 Nov 05 02:40:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_24(8161782) done Nov 05 02:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3116(8130969) WEXITSTATUS 1 Nov 05 02:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3116(8130969) done Nov 05 02:40:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_381(8162226) WEXITSTATUS 0 Nov 05 02:40:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_381(8162226) done Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6258(8163540) in scavenge on r907u09n02 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6259(8163541) in scavenge on r912u05n04 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6260(8163542) in scavenge on r917u01n02 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6261(8163543) in scavenge on r802u02n01 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6262(8163544) in scavenge on r802u02n02 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6263(8163545) in scavenge on r802u11n03 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6264(8163546) in scavenge on r802u30n01 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6265(8163547) in scavenge on r815u03n09 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6266(8163548) in scavenge on r816u23n08 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6267(8163549) in scavenge on r816u29n08 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6268(8163550) in scavenge on r816u35n11 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6269(8163551) in scavenge on r816u35n11 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6270(8163552) in scavenge on r816u35n11 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6271(8163553) in scavenge on r808u11n01 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6272(8163554) in scavenge on r806u23n01 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6273(8163555) in scavenge on r807u03n03 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6274(8163556) in scavenge on r807u08n01 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6275(8163557) in scavenge on r912u32n04 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_838(7573007) in scavenge on r815u03n09 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_69(8115539) in scavenge on r805u14n01 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_590(8116063) in scavenge on r913u05n04 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_443(8114710) in scavenge on r816u23n06 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_637(7572806) in scavenge on r816u35n04 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_70(8161828) WEXITSTATUS 0 Nov 05 02:40:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_70(8161828) done Nov 05 02:40:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_441(8162297) WEXITSTATUS 0 Nov 05 02:40:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_441(8162297) done Nov 05 02:40:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159886 WEXITSTATUS 0 Nov 05 02:40:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159886 done Nov 05 02:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1326(8135423) WEXITSTATUS 0 Nov 05 02:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1326(8135423) done Nov 05 02:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2538(8152805) WEXITSTATUS 0 Nov 05 02:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2538(8152805) done Nov 05 02:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_142(8161920) WEXITSTATUS 0 Nov 05 02:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_142(8161920) done Nov 05 02:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_212(8162007) WEXITSTATUS 0 Nov 05 02:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_212(8162007) done Nov 05 02:41:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_151(8161929) WEXITSTATUS 0 Nov 05 02:41:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_151(8161929) done Nov 05 02:41:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_71(8161829) WEXITSTATUS 0 Nov 05 02:41:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_71(8161829) done Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_206(8162001) WEXITSTATUS 0 Nov 05 02:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_206(8162001) done Nov 05 02:41:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5982(8162982) WEXITSTATUS 1 Nov 05 02:41:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5982(8162982) done Nov 05 02:41:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159105 WEXITSTATUS 0 Nov 05 02:41:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159105 done Nov 05 02:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5789(8162744) WEXITSTATUS 1 Nov 05 02:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5789(8162744) done Nov 05 02:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_975(8128352) WEXITSTATUS 0 Nov 05 02:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_975(8128352) done Nov 05 02:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_4(8162884) WEXITSTATUS 0 Nov 05 02:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_4(8162884) done Nov 05 02:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6042(8163064) WEXITSTATUS 1 Nov 05 02:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6042(8163064) done Nov 05 02:41:15 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4776(8157458) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:41:15 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5551(8161041) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:41:15 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5722(8162524) has been requeued to reclaim resources for JobId=Invalid Nov 05 02:41:16 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4776(8157458) Nov 05 02:41:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5551(8161041) Nov 05 02:41:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5722(8162524) Nov 05 02:41:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_283(8162088) WEXITSTATUS 0 Nov 05 02:41:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_283(8162088) done Nov 05 02:41:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3709(8163332) WEXITSTATUS 0 Nov 05 02:41:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3709(8163332) done Nov 05 02:41:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1430(8135963) WEXITSTATUS 0 Nov 05 02:41:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1430(8135963) done Nov 05 02:41:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_331(8162145) WEXITSTATUS 0 Nov 05 02:41:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_331(8162145) done Nov 05 02:41:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_137(8161915) WEXITSTATUS 0 Nov 05 02:41:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_137(8161915) done Nov 05 02:41:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162677 NodeList=r806u04n02 #CPUs=16 Partition=day Nov 05 02:41:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5704(8162417) WEXITSTATUS 1 Nov 05 02:41:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5704(8162417) done Nov 05 02:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6024(8163034) WEXITSTATUS 1 Nov 05 02:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6024(8163034) done Nov 05 02:41:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_215(8162010) WEXITSTATUS 0 Nov 05 02:41:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_215(8162010) done Nov 05 02:41:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162184 WEXITSTATUS 0 Nov 05 02:41:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162184 done Nov 05 02:41:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_213(8162008) WEXITSTATUS 0 Nov 05 02:41:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_213(8162008) done Nov 05 02:41:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6218(8163370) WEXITSTATUS 1 Nov 05 02:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6218(8163370) done Nov 05 02:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1394(8135803) WEXITSTATUS 0 Nov 05 02:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1394(8135803) done Nov 05 02:41:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1473(8138474) WEXITSTATUS 0 Nov 05 02:41:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1473(8138474) done Nov 05 02:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_209(8162004) WEXITSTATUS 0 Nov 05 02:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_209(8162004) done Nov 05 02:42:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_271(8162076) WEXITSTATUS 0 Nov 05 02:42:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_271(8162076) done Nov 05 02:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_433(8162289) WEXITSTATUS 0 Nov 05 02:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_433(8162289) done Nov 05 02:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_179(8161974) WEXITSTATUS 0 Nov 05 02:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_179(8161974) done Nov 05 02:42:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_127(8161905) WEXITSTATUS 0 Nov 05 02:42:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_127(8161905) done Nov 05 02:42:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_474(8162344) WEXITSTATUS 0 Nov 05 02:42:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_474(8162344) done Nov 05 02:42:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5882(8162858) WEXITSTATUS 1 Nov 05 02:42:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5882(8162858) done Nov 05 02:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159926 WEXITSTATUS 0 Nov 05 02:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159926 done Nov 05 02:42:14 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2269(8126183) Nov 05 02:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162186 WEXITSTATUS 0 Nov 05 02:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162186 done Nov 05 02:42:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:20 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:25 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:26 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_440(8162296) WEXITSTATUS 0 Nov 05 02:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_440(8162296) done Nov 05 02:42:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162678 NodeList=r909u29n02 #CPUs=16 Partition=day Nov 05 02:42:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5926(8162921) WEXITSTATUS 1 Nov 05 02:42:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5926(8162921) done Nov 05 02:42:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_19(8161777) WEXITSTATUS 0 Nov 05 02:42:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_19(8161777) done Nov 05 02:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_4(8162890) WEXITSTATUS 0 Nov 05 02:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_4(8162890) done Nov 05 02:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159921 WEXITSTATUS 0 Nov 05 02:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159921 done Nov 05 02:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_205(8162000) WEXITSTATUS 0 Nov 05 02:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_205(8162000) done Nov 05 02:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_21(8160520) WEXITSTATUS 0 Nov 05 02:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_21(8160520) done Nov 05 02:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_29(8161787) WEXITSTATUS 0 Nov 05 02:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_29(8161787) done Nov 05 02:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_180(8161975) WEXITSTATUS 0 Nov 05 02:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_180(8161975) done Nov 05 02:42:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_143(8161921) WEXITSTATUS 0 Nov 05 02:42:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_143(8161921) done Nov 05 02:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_22(8161780) WEXITSTATUS 0 Nov 05 02:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_22(8161780) done Nov 05 02:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_223(8162018) WEXITSTATUS 0 Nov 05 02:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_223(8162018) done Nov 05 02:43:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_147(8161925) WEXITSTATUS 0 Nov 05 02:43:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_147(8161925) done Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:43:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_131(8161909) WEXITSTATUS 0 Nov 05 02:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_131(8161909) done Nov 05 02:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1358(8135621) WEXITSTATUS 0 Nov 05 02:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1358(8135621) done Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_142(8163558) in pi_ohern on r917u13n03 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_143(8163559) in pi_ohern on r917u13n03 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_144(8163560) in pi_ohern on r917u13n04 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_145(8163561) in pi_ohern on r917u13n04 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_146(8163562) in pi_ohern on r917u13n04 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_147(8163563) in pi_ohern on r917u13n04 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_148(8163564) in pi_ohern on r815u03n11 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_149(8163565) in pi_ohern on r815u03n11 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_150(8163566) in pi_ohern on r815u03n12 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_151(8163567) in pi_ohern on r815u03n12 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_152(8163568) in pi_ohern on r815u03n12 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_153(8163569) in pi_ohern on r815u03n12 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_154(8163570) in pi_ohern on r911u32n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_155(8163571) in pi_ohern on r911u32n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_156(8163572) in pi_ohern on r911u32n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_157(8163573) in pi_ohern on r911u32n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_158(8163574) in pi_ohern on r913u32n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_159(8163575) in pi_ohern on r913u32n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_160(8163576) in pi_ohern on r913u32n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_161(8163577) in pi_ohern on r807u22n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_162(8163578) in pi_ohern on r807u22n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_163(8163579) in pi_ohern on r807u22n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_164(8163580) in pi_ohern on r807u24n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_165(8163581) in pi_ohern on r903u12n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_166(8163582) in pi_ohern on r903u12n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_167(8163583) in pi_ohern on r903u24n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_168(8163584) in pi_ohern on r903u24n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_169(8163585) in pi_ohern on r903u24n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_170(8163586) in pi_ohern on r903u26n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_171(8163587) in pi_ohern on r903u26n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162679 in day on r914u05n03 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3722(8163588) in gpu on r808u11n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154707 in day on r919u09n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1957(8163589) in day on r914u05n02 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1958(8163590) in day on r909u04n02 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1959(8163591) in day on r909u13n04 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1960(8163592) in day on r918u01n01 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1961(8163593) in day on r918u36n02 Nov 05 02:43:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1962(8163594) in day on r816u15n03 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6276(8163595) in scavenge on r801u27n02 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6277(8163596) in scavenge on r815u15n11 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6278(8163597) in scavenge on r815u23n12 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6279(8163598) in scavenge on r816u23n11 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6280(8163599) in scavenge on r816u29n08 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6281(8163600) in scavenge on r816u35n12 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6282(8163601) in scavenge on r904u04n03 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6283(8163602) in scavenge on r904u10n03 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6284(8163603) in scavenge on r805u25n04 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6285(8163604) in scavenge on r805u27n02 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6286(8163605) in scavenge on r918u05n01 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6287(8163606) in scavenge on r918u05n01 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_920(7573089) in scavenge on r806u04n02 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1006(7573175) in scavenge on r909u07n02 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_308(8114574) in scavenge on r909u07n02 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_322(8114589) in scavenge on r815u29n04 Nov 05 02:43:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_148(8115618) in scavenge on r816u15n03 Nov 05 02:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_345(8162159) WEXITSTATUS 0 Nov 05 02:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_345(8162159) done Nov 05 02:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1500(8142113) WEXITSTATUS 0 Nov 05 02:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1500(8142113) done Nov 05 02:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5933(8162928) WEXITSTATUS 1 Nov 05 02:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5933(8162928) done Nov 05 02:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_136(8161914) WEXITSTATUS 0 Nov 05 02:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_136(8161914) done Nov 05 02:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_16(8161774) WEXITSTATUS 0 Nov 05 02:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_16(8161774) done Nov 05 02:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_473(8162343) WEXITSTATUS 0 Nov 05 02:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_473(8162343) done Nov 05 02:43:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162317 WEXITSTATUS 0 Nov 05 02:43:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162317 done Nov 05 02:43:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_342(8162156) WEXITSTATUS 0 Nov 05 02:43:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_342(8162156) done Nov 05 02:43:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_222(8162017) WEXITSTATUS 0 Nov 05 02:43:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_222(8162017) done Nov 05 02:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5928(8162923) WEXITSTATUS 1 Nov 05 02:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5928(8162923) done Nov 05 02:43:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162318 WEXITSTATUS 0 Nov 05 02:43:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162318 done Nov 05 02:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_178(8161973) WEXITSTATUS 0 Nov 05 02:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_178(8161973) done Nov 05 02:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_139(8161917) WEXITSTATUS 0 Nov 05 02:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_139(8161917) done Nov 05 02:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162187 WEXITSTATUS 0 Nov 05 02:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162187 done Nov 05 02:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5524(8160703) WEXITSTATUS 1 Nov 05 02:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5524(8160703) done Nov 05 02:44:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3710(8163333) WEXITSTATUS 0 Nov 05 02:44:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3710(8163333) done Nov 05 02:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6039(8163061) WEXITSTATUS 1 Nov 05 02:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6039(8163061) done Nov 05 02:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_30(8161788) WEXITSTATUS 0 Nov 05 02:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_30(8161788) done Nov 05 02:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162695 WEXITSTATUS 0 Nov 05 02:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162695 done Nov 05 02:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_274(8162079) WEXITSTATUS 0 Nov 05 02:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_274(8162079) done Nov 05 02:44:15 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162696 NodeList=r801u04n01 #CPUs=12 Partition=pi_esi Nov 05 02:44:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_273(8162078) WEXITSTATUS 0 Nov 05 02:44:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_273(8162078) done Nov 05 02:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_110(8161888) WEXITSTATUS 0 Nov 05 02:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_110(8161888) done Nov 05 02:44:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6167(8163245) WEXITSTATUS 1 Nov 05 02:44:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6167(8163245) done Nov 05 02:44:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_153(8161931) WEXITSTATUS 0 Nov 05 02:44:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_153(8161931) done Nov 05 02:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154641 WEXITSTATUS 0 Nov 05 02:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154641 done Nov 05 02:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_265(8162070) WEXITSTATUS 0 Nov 05 02:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_265(8162070) done Nov 05 02:44:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1574(8150817) WEXITSTATUS 0 Nov 05 02:44:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1574(8150817) done Nov 05 02:44:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1333(8135466) WEXITSTATUS 0 Nov 05 02:44:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1333(8135466) done Nov 05 02:44:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5595(8161671) WEXITSTATUS 1 Nov 05 02:44:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5595(8161671) done Nov 05 02:44:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_152(8161930) WEXITSTATUS 0 Nov 05 02:44:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_152(8161930) done Nov 05 02:44:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2271(8126188) Nov 05 02:44:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2273(8126190) Nov 05 02:44:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_13(8161771) WEXITSTATUS 0 Nov 05 02:44:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_13(8161771) done Nov 05 02:44:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_269(8162074) WEXITSTATUS 0 Nov 05 02:44:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_269(8162074) done Nov 05 02:44:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_267(8162072) WEXITSTATUS 0 Nov 05 02:44:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_267(8162072) done Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:45:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6242(8163448) WEXITSTATUS 1 Nov 05 02:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6242(8163448) done Nov 05 02:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_217(8162012) WEXITSTATUS 0 Nov 05 02:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_217(8162012) done Nov 05 02:45:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_214(8162009) WEXITSTATUS 0 Nov 05 02:45:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_214(8162009) done Nov 05 02:45:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_219(8162014) WEXITSTATUS 0 Nov 05 02:45:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_219(8162014) done Nov 05 02:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_11(8161769) WEXITSTATUS 0 Nov 05 02:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_11(8161769) done Nov 05 02:45:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_306(8162120) WEXITSTATUS 0 Nov 05 02:45:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_306(8162120) done Nov 05 02:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3699(8163229) WEXITSTATUS 0 Nov 05 02:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3699(8163229) done Nov 05 02:45:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_149(8161927) WEXITSTATUS 0 Nov 05 02:45:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_149(8161927) done Nov 05 02:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1602(8153020) WEXITSTATUS 0 Nov 05 02:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1602(8153020) done Nov 05 02:45:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6100(8163138) WEXITSTATUS 1 Nov 05 02:45:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6100(8163138) done Nov 05 02:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6013(8163018) WEXITSTATUS 1 Nov 05 02:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6013(8163018) done Nov 05 02:45:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_305(8162119) WEXITSTATUS 0 Nov 05 02:45:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_305(8162119) done Nov 05 02:45:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_218(8162013) WEXITSTATUS 0 Nov 05 02:45:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_218(8162013) done Nov 05 02:45:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5132(8158472) WEXITSTATUS 1 Nov 05 02:45:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5132(8158472) done Nov 05 02:45:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_268(8162073) WEXITSTATUS 0 Nov 05 02:45:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_268(8162073) done Nov 05 02:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_48(8158727) WEXITSTATUS 0 Nov 05 02:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_48(8158727) done Nov 05 02:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1455(8137180) WEXITSTATUS 0 Nov 05 02:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1455(8137180) done Nov 05 02:46:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_244(8162049) WEXITSTATUS 0 Nov 05 02:46:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_244(8162049) done Nov 05 02:46:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_12(8161770) WEXITSTATUS 0 Nov 05 02:46:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_12(8161770) done Nov 05 02:46:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_278(8162083) WEXITSTATUS 0 Nov 05 02:46:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_278(8162083) done Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_172(8163607) in pi_ohern on r917u13n02 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_173(8163608) in pi_ohern on r917u13n02 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_174(8163609) in pi_ohern on r917u13n02 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_175(8163610) in pi_ohern on r917u13n02 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_176(8163611) in pi_ohern on r917u13n03 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_177(8163612) in pi_ohern on r917u13n04 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_178(8163613) in pi_ohern on r917u13n04 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_179(8163614) in pi_ohern on r917u13n04 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_180(8163615) in pi_ohern on r917u13n04 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_181(8163616) in pi_ohern on r917u29n04 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_182(8163617) in pi_ohern on r815u03n12 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_183(8163618) in pi_ohern on r903u24n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_184(8163619) in pi_ohern on r903u24n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_185(8163620) in pi_ohern on r903u24n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_186(8163621) in pi_ohern on r903u24n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_187(8163622) in pi_ohern on r903u24n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_188(8163623) in pi_ohern on r911u32n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_189(8163624) in pi_ohern on r807u22n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_190(8163625) in pi_ohern on r807u22n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_191(8163626) in pi_ohern on r807u22n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_192(8163627) in pi_ohern on r807u24n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_193(8163628) in pi_ohern on r807u24n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_194(8163629) in pi_ohern on r807u24n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_195(8163630) in pi_ohern on r913u32n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_196(8163631) in pi_ohern on r913u32n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_197(8163632) in pi_ohern on r913u32n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_198(8163633) in pi_ohern on r913u32n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_199(8163634) in pi_ohern on r913u32n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_200(8163635) in pi_ohern on r913u32n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_201(8163636) in pi_ohern on r913u35n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3723(8163637) in gpu on r808u17n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3724(8163638) in gpu on r4513u26n01 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154710 in day on r816u03n05 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_277(8162082) WEXITSTATUS 0 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_277(8162082) done Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_181(8161976) WEXITSTATUS 0 Nov 05 02:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_181(8161976) done Nov 05 02:46:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1963(8163639) in day on r805u27n01 Nov 05 02:46:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1964(8163640) in day on r815u23n06 Nov 05 02:46:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:46:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5592(8161555) WEXITSTATUS 1 Nov 05 02:46:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5592(8161555) done Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6288(8163641) in scavenge on r908u28n03 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6289(8163642) in scavenge on r907u01n04 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6290(8163643) in scavenge on r907u05n03 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6291(8163644) in scavenge on r904u01n03 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6292(8163645) in scavenge on r802u09n03 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6293(8163646) in scavenge on r802u32n03 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6294(8163647) in scavenge on r815u23n07 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6295(8163648) in scavenge on r815u23n07 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6296(8163649) in scavenge on r816u15n11 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6297(8163650) in scavenge on r816u29n07 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6298(8163651) in scavenge on r906u28n01 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6299(8163652) in scavenge on r904u10n01 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6300(8163653) in scavenge on r904u10n03 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_972(7573141) in scavenge on r918u01n02 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_8(8114274) in scavenge on r918u01n02 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_73(8161831) WEXITSTATUS 0 Nov 05 02:46:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_73(8161831) done Nov 05 02:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159924 WEXITSTATUS 0 Nov 05 02:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159924 done Nov 05 02:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_287(8162092) WEXITSTATUS 0 Nov 05 02:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_287(8162092) done Nov 05 02:46:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1269(8135207) WEXITSTATUS 0 Nov 05 02:46:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1269(8135207) done Nov 05 02:46:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_148(8161926) WEXITSTATUS 0 Nov 05 02:46:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_148(8161926) done Nov 05 02:46:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5686(8162308) WEXITSTATUS 1 Nov 05 02:46:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5686(8162308) done Nov 05 02:46:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_100(8161878) WEXITSTATUS 0 Nov 05 02:46:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_100(8161878) done Nov 05 02:46:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_442(8162298) WEXITSTATUS 0 Nov 05 02:46:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_442(8162298) done Nov 05 02:46:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_280(8162085) WEXITSTATUS 0 Nov 05 02:46:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_280(8162085) done Nov 05 02:46:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6289(8163642) WEXITSTATUS 1 Nov 05 02:46:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6289(8163642) done Nov 05 02:46:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_4(8161762) WEXITSTATUS 0 Nov 05 02:46:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_4(8161762) done Nov 05 02:46:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162680 NodeList=r816u03n03 #CPUs=16 Partition=day Nov 05 02:46:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_341(8162155) WEXITSTATUS 0 Nov 05 02:46:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_341(8162155) done Nov 05 02:46:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_272(8162077) WEXITSTATUS 0 Nov 05 02:46:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_272(8162077) done Nov 05 02:46:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_224(8162019) WEXITSTATUS 0 Nov 05 02:46:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_224(8162019) done Nov 05 02:46:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_225(8162020) WEXITSTATUS 0 Nov 05 02:46:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_225(8162020) done Nov 05 02:46:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_18(8161776) WEXITSTATUS 0 Nov 05 02:46:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_18(8161776) done Nov 05 02:47:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_266(8162071) WEXITSTATUS 0 Nov 05 02:47:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_266(8162071) done Nov 05 02:47:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6266(8163548) WEXITSTATUS 1 Nov 05 02:47:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6266(8163548) done Nov 05 02:47:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_92(8161870) WEXITSTATUS 0 Nov 05 02:47:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_92(8161870) done Nov 05 02:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1482(8138809) WEXITSTATUS 0 Nov 05 02:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1482(8138809) done Nov 05 02:47:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_284(8162089) WEXITSTATUS 0 Nov 05 02:47:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_284(8162089) done Nov 05 02:47:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_174(8114440) Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:47:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:47:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1491(8140410) WEXITSTATUS 0 Nov 05 02:47:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1491(8140410) done Nov 05 02:47:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5801(8162760) WEXITSTATUS 1 Nov 05 02:47:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5801(8162760) done Nov 05 02:47:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_521(8162407) WEXITSTATUS 0 Nov 05 02:47:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_521(8162407) done Nov 05 02:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4731(8156860) WEXITSTATUS 1 Nov 05 02:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4731(8156860) done Nov 05 02:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162188 WEXITSTATUS 0 Nov 05 02:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162188 done Nov 05 02:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_281(8162086) WEXITSTATUS 0 Nov 05 02:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_281(8162086) done Nov 05 02:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_216(8162011) WEXITSTATUS 0 Nov 05 02:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_216(8162011) done Nov 05 02:47:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_6(8161764) WEXITSTATUS 0 Nov 05 02:47:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_6(8161764) done Nov 05 02:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_338(8162152) WEXITSTATUS 0 Nov 05 02:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_338(8162152) done Nov 05 02:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_337(8162151) WEXITSTATUS 0 Nov 05 02:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_337(8162151) done Nov 05 02:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_99(8161877) WEXITSTATUS 0 Nov 05 02:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_99(8161877) done Nov 05 02:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_17(8161775) WEXITSTATUS 0 Nov 05 02:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_17(8161775) done Nov 05 02:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_210(8162005) WEXITSTATUS 0 Nov 05 02:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_210(8162005) done Nov 05 02:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_5(8161763) WEXITSTATUS 0 Nov 05 02:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_5(8161763) done Nov 05 02:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_15(8161773) WEXITSTATUS 0 Nov 05 02:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_15(8161773) done Nov 05 02:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_9(8161767) WEXITSTATUS 0 Nov 05 02:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_9(8161767) done Nov 05 02:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6000(8163005) WEXITSTATUS 1 Nov 05 02:47:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6000(8163005) done Nov 05 02:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5245(8158652) WEXITSTATUS 1 Nov 05 02:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5245(8158652) done Nov 05 02:47:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_8(8161766) WEXITSTATUS 0 Nov 05 02:47:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_8(8161766) done Nov 05 02:47:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_270(8162075) WEXITSTATUS 0 Nov 05 02:47:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_270(8162075) done Nov 05 02:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_31(8158663) WEXITSTATUS 0 Nov 05 02:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_31(8158663) done Nov 05 02:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1391(8135800) WEXITSTATUS 0 Nov 05 02:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1391(8135800) done Nov 05 02:47:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5940(8162935) WEXITSTATUS 1 Nov 05 02:47:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5940(8162935) done Nov 05 02:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5892(8162868) WEXITSTATUS 1 Nov 05 02:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5892(8162868) done Nov 05 02:48:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_156(8161951) WEXITSTATUS 0 Nov 05 02:48:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_156(8161951) done Nov 05 02:48:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_282(8162087) WEXITSTATUS 0 Nov 05 02:48:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_282(8162087) done Nov 05 02:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1499(8142112) WEXITSTATUS 0 Nov 05 02:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1499(8142112) done Nov 05 02:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3724(8163638) WEXITSTATUS 0 Nov 05 02:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3724(8163638) done Nov 05 02:48:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6002(8163007) WEXITSTATUS 1 Nov 05 02:48:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6002(8163007) done Nov 05 02:48:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1441(8135974) WEXITSTATUS 0 Nov 05 02:48:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1441(8135974) done Nov 05 02:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_138(8161916) WEXITSTATUS 0 Nov 05 02:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_138(8161916) done Nov 05 02:48:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146704 Nov 05 02:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5812(8162772) WEXITSTATUS 1 Nov 05 02:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5812(8162772) done Nov 05 02:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6199(8163321) WEXITSTATUS 1 Nov 05 02:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6199(8163321) done Nov 05 02:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_279(8162084) WEXITSTATUS 0 Nov 05 02:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_279(8162084) done Nov 05 02:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_135(8161913) WEXITSTATUS 0 Nov 05 02:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_135(8161913) done Nov 05 02:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_364(8162209) WEXITSTATUS 0 Nov 05 02:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_364(8162209) done Nov 05 02:48:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6228(8163409) WEXITSTATUS 1 Nov 05 02:48:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6228(8163409) done Nov 05 02:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1466(8138361) WEXITSTATUS 0 Nov 05 02:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1466(8138361) done Nov 05 02:48:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_398(8162243) WEXITSTATUS 0 Nov 05 02:48:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_398(8162243) done Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_202(8163654) in pi_ohern on r801u32n02 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_203(8163655) in pi_ohern on r801u32n02 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_204(8163656) in pi_ohern on r917u13n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_205(8163657) in pi_ohern on r917u13n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_206(8163658) in pi_ohern on r917u13n02 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_207(8163659) in pi_ohern on r917u13n02 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_208(8163660) in pi_ohern on r917u13n02 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_209(8163661) in pi_ohern on r917u13n02 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_210(8163662) in pi_ohern on r917u13n02 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_211(8163663) in pi_ohern on r917u13n03 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_212(8163664) in pi_ohern on r917u13n03 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_213(8163665) in pi_ohern on r917u13n04 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_214(8163666) in pi_ohern on r917u13n04 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_215(8163667) in pi_ohern on r917u13n04 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_216(8163668) in pi_ohern on r815u03n12 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_217(8163669) in pi_ohern on r815u03n12 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_218(8163670) in pi_ohern on r911u32n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_219(8163671) in pi_ohern on r911u32n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_220(8163672) in pi_ohern on r807u22n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_221(8163673) in pi_ohern on r807u22n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_222(8163674) in pi_ohern on r807u22n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_223(8163675) in pi_ohern on r807u22n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_224(8163676) in pi_ohern on r807u22n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_225(8163677) in pi_ohern on r807u24n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_226(8163678) in pi_ohern on r913u32n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_227(8163679) in pi_ohern on r913u32n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_228(8163680) in pi_ohern on r913u32n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_229(8163681) in pi_ohern on r913u32n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_230(8163682) in pi_ohern on r913u35n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_231(8163683) in pi_ohern on r913u35n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_232(8163684) in pi_ohern on r903u12n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_233(8163685) in pi_ohern on r903u24n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_234(8163686) in pi_ohern on r903u24n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_235(8163687) in pi_ohern on r903u24n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_236(8163688) in pi_ohern on r903u24n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_237(8163689) in pi_ohern on r903u24n01 Nov 05 02:48:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_238(8163690) in pi_ohern on r903u24n01 Nov 05 02:48:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_239(8163691) in pi_ohern on r903u26n01 Nov 05 02:48:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3725(8163692) in gpu on r4513u26n01 Nov 05 02:48:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1965(8163693) in day on r909u13n01 Nov 05 02:48:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1966(8163694) in day on r918u36n01 Nov 05 02:48:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1967(8163695) in day on r919u01n01 Nov 05 02:48:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1968(8163696) in day on r816u15n04 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6301(8163697) in scavenge on r907u01n04 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6302(8163698) in scavenge on r802u09n03 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6303(8163699) in scavenge on r915u32n01 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6304(8163700) in scavenge on r815u15n12 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6305(8163701) in scavenge on r816u03n10 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6306(8163702) in scavenge on r816u23n08 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6307(8163703) in scavenge on r816u23n11 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6308(8163704) in scavenge on r904u07n03 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6309(8163705) in scavenge on r913u01n02 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6310(8163706) in scavenge on r805u04n02 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6311(8163707) in scavenge on r805u09n02 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6312(8163708) in scavenge on r805u25n01 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6313(8163709) in scavenge on r918u09n04 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_525(7572694) in scavenge on r815u09n06 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_781(8115050) in scavenge on r815u09n06 Nov 05 02:48:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_790(8115059) in scavenge on r816u29n05 Nov 05 02:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_290(8162095) WEXITSTATUS 0 Nov 05 02:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_290(8162095) done Nov 05 02:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_20(8161778) WEXITSTATUS 0 Nov 05 02:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_20(8161778) done Nov 05 02:49:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5671(8162163) WEXITSTATUS 1 Nov 05 02:49:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5671(8162163) done Nov 05 02:49:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_276(8162081) WEXITSTATUS 0 Nov 05 02:49:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_276(8162081) done Nov 05 02:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_343(8162157) WEXITSTATUS 0 Nov 05 02:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_343(8162157) done Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162190 WEXITSTATUS 0 Nov 05 02:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162190 done Nov 05 02:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_236(8162041) WEXITSTATUS 0 Nov 05 02:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_236(8162041) done Nov 05 02:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_335(8162149) WEXITSTATUS 0 Nov 05 02:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_335(8162149) done Nov 05 02:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1268(8135206) WEXITSTATUS 0 Nov 05 02:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1268(8135206) done Nov 05 02:49:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1354(8135531) WEXITSTATUS 0 Nov 05 02:49:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1354(8135531) done Nov 05 02:49:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_346(8162160) WEXITSTATUS 0 Nov 05 02:49:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_346(8162160) done Nov 05 02:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5855(8162826) WEXITSTATUS 1 Nov 05 02:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5855(8162826) done Nov 05 02:49:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_14(8161772) WEXITSTATUS 0 Nov 05 02:49:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_14(8161772) done Nov 05 02:49:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_157(8161952) WEXITSTATUS 0 Nov 05 02:49:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_157(8161952) done Nov 05 02:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_161(8161956) WEXITSTATUS 0 Nov 05 02:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_161(8161956) done Nov 05 02:49:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6012(8163017) WEXITSTATUS 1 Nov 05 02:49:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6012(8163017) done Nov 05 02:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_177(8161972) WEXITSTATUS 0 Nov 05 02:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_177(8161972) done Nov 05 02:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1320(8135411) WEXITSTATUS 0 Nov 05 02:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1320(8135411) done Nov 05 02:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_336(8162150) WEXITSTATUS 0 Nov 05 02:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_336(8162150) done Nov 05 02:49:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162658 WEXITSTATUS 0 Nov 05 02:49:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162658 done Nov 05 02:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6007(8163012) WEXITSTATUS 1 Nov 05 02:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6007(8163012) done Nov 05 02:50:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5775(8162717) WEXITSTATUS 1 Nov 05 02:50:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5775(8162717) done Nov 05 02:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_90(8161868) WEXITSTATUS 0 Nov 05 02:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_90(8161868) done Nov 05 02:50:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1110(8131602) WEXITSTATUS 0 Nov 05 02:50:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1110(8131602) done Nov 05 02:50:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_84(8161862) WEXITSTATUS 0 Nov 05 02:50:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_84(8161862) done Nov 05 02:50:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_340(8162154) WEXITSTATUS 0 Nov 05 02:50:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_340(8162154) done Nov 05 02:50:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_88(8161866) WEXITSTATUS 0 Nov 05 02:50:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_88(8161866) done Nov 05 02:50:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_93(8161871) WEXITSTATUS 0 Nov 05 02:50:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_93(8161871) done Nov 05 02:50:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_40(8158688) WEXITSTATUS 0 Nov 05 02:50:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_40(8158688) done Nov 05 02:50:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162681 NodeList=r909u04n04 #CPUs=16 Partition=day Nov 05 02:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2362(8126371) WEXITSTATUS 0 Nov 05 02:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2362(8126371) done Nov 05 02:50:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_339(8162153) WEXITSTATUS 0 Nov 05 02:50:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_339(8162153) done Nov 05 02:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_158(8161953) WEXITSTATUS 0 Nov 05 02:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_158(8161953) done Nov 05 02:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6280(8163599) WEXITSTATUS 1 Nov 05 02:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6280(8163599) done Nov 05 02:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_234(8162039) WEXITSTATUS 0 Nov 05 02:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_234(8162039) done Nov 05 02:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1510(8143366) WEXITSTATUS 0 Nov 05 02:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1510(8143366) done Nov 05 02:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6259(8163541) WEXITSTATUS 1 Nov 05 02:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6259(8163541) done Nov 05 02:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_165(8161960) WEXITSTATUS 0 Nov 05 02:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_165(8161960) done Nov 05 02:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_192(8161987) WEXITSTATUS 0 Nov 05 02:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_192(8161987) done Nov 05 02:51:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_3(8161761) WEXITSTATUS 0 Nov 05 02:51:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_3(8161761) done Nov 05 02:51:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1411(8135944) WEXITSTATUS 0 Nov 05 02:51:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1411(8135944) done Nov 05 02:51:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5985(8162985) WEXITSTATUS 1 Nov 05 02:51:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5985(8162985) done Nov 05 02:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_400(8162245) WEXITSTATUS 0 Nov 05 02:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_400(8162245) done Nov 05 02:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6240(8163446) WEXITSTATUS 1 Nov 05 02:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6240(8163446) done Nov 05 02:51:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5992(8162992) WEXITSTATUS 1 Nov 05 02:51:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5992(8162992) done Nov 05 02:51:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_399(8162244) WEXITSTATUS 0 Nov 05 02:51:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_399(8162244) done Nov 05 02:51:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_288(8162093) WEXITSTATUS 0 Nov 05 02:51:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_288(8162093) done Nov 05 02:51:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1478(8138592) WEXITSTATUS 0 Nov 05 02:51:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1478(8138592) done Nov 05 02:51:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_162(8161957) WEXITSTATUS 0 Nov 05 02:51:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_162(8161957) done Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:51:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:51:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_85(8161863) WEXITSTATUS 0 Nov 05 02:51:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_85(8161863) done Nov 05 02:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_233(8162038) WEXITSTATUS 0 Nov 05 02:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_233(8162038) done Nov 05 02:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2339(8152256) WEXITSTATUS 0 Nov 05 02:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2339(8152256) done Nov 05 02:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_108(8161886) WEXITSTATUS 0 Nov 05 02:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_108(8161886) done Nov 05 02:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6306(8163702) WEXITSTATUS 1 Nov 05 02:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6306(8163702) done Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_240(8163710) in pi_ohern on r801u32n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_241(8163711) in pi_ohern on r801u32n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_242(8163712) in pi_ohern on r801u32n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_243(8163713) in pi_ohern on r801u32n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_244(8163714) in pi_ohern on r801u32n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_245(8163715) in pi_ohern on r917u13n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_246(8163716) in pi_ohern on r917u13n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_247(8163717) in pi_ohern on r917u13n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_248(8163718) in pi_ohern on r917u13n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_249(8163719) in pi_ohern on r917u13n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_250(8163720) in pi_ohern on r917u13n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_251(8163721) in pi_ohern on r917u13n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_252(8163722) in pi_ohern on r917u13n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_253(8163723) in pi_ohern on r917u13n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_254(8163724) in pi_ohern on r917u13n02 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_255(8163725) in pi_ohern on r917u13n03 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_256(8163726) in pi_ohern on r917u13n03 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_257(8163727) in pi_ohern on r917u29n03 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_258(8163728) in pi_ohern on r917u29n04 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_259(8163729) in pi_ohern on r903u24n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_260(8163730) in pi_ohern on r903u24n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_261(8163731) in pi_ohern on r913u35n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_262(8163732) in pi_ohern on r913u35n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_263(8163733) in pi_ohern on r913u35n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_264(8163734) in pi_ohern on r913u35n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_265(8163735) in pi_ohern on r807u22n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_266(8163736) in pi_ohern on r807u22n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_267(8163737) in pi_ohern on r807u24n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_268(8163738) in pi_ohern on r807u24n01 Nov 05 02:51:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_269(8163739) in pi_ohern on r807u24n01 Nov 05 02:51:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1969(8163740) in day on r909u29n03 Nov 05 02:51:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1970(8163741) in day on r918u05n01 Nov 05 02:51:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1971(8163742) in day on r919u32n02 Nov 05 02:51:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1972(8163743) in day on r815u35n01 Nov 05 02:51:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:51:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_91(8161869) WEXITSTATUS 0 Nov 05 02:51:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_91(8161869) done Nov 05 02:51:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_104(8161882) WEXITSTATUS 0 Nov 05 02:51:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_104(8161882) done Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_169(8161964) WEXITSTATUS 0 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_169(8161964) done Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6314(8163744) in scavenge on r802u04n04 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6315(8163745) in scavenge on r802u04n04 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6316(8163746) in scavenge on r912u05n04 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6317(8163747) in scavenge on r816u23n08 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6318(8163748) in scavenge on r816u29n08 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6319(8163749) in scavenge on r816u29n10 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6320(8163750) in scavenge on r816u35n10 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6321(8163751) in scavenge on r909u01n02 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6322(8163752) in scavenge on r909u01n04 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6323(8163753) in scavenge on r909u04n04 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6324(8163754) in scavenge on r909u07n04 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6325(8163755) in scavenge on r909u07n04 Nov 05 02:51:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1058(7573227) in scavenge on r815u35n01 Nov 05 02:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6146(8163193) WEXITSTATUS 1 Nov 05 02:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6146(8163193) done Nov 05 02:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6174(8163260) WEXITSTATUS 1 Nov 05 02:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6174(8163260) done Nov 05 02:51:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_447(8162303) WEXITSTATUS 0 Nov 05 02:51:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_447(8162303) done Nov 05 02:51:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_478(8162348) WEXITSTATUS 0 Nov 05 02:51:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_478(8162348) done Nov 05 02:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_89(8161867) WEXITSTATUS 0 Nov 05 02:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_89(8161867) done Nov 05 02:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_82(8161860) WEXITSTATUS 0 Nov 05 02:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_82(8161860) done Nov 05 02:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_34(8161792) WEXITSTATUS 0 Nov 05 02:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_34(8161792) done Nov 05 02:51:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_101(8161879) WEXITSTATUS 0 Nov 05 02:51:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_101(8161879) done Nov 05 02:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6275(8163557) WEXITSTATUS 1 Nov 05 02:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6275(8163557) done Nov 05 02:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_21(8161779) WEXITSTATUS 0 Nov 05 02:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_21(8161779) done Nov 05 02:52:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_163(8161958) WEXITSTATUS 0 Nov 05 02:52:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_163(8161958) done Nov 05 02:52:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_7(8161765) WEXITSTATUS 0 Nov 05 02:52:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_7(8161765) done Nov 05 02:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_32(8161790) WEXITSTATUS 0 Nov 05 02:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_32(8161790) done Nov 05 02:52:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_477(8162347) WEXITSTATUS 0 Nov 05 02:52:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_477(8162347) done Nov 05 02:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_291(8162096) WEXITSTATUS 0 Nov 05 02:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_291(8162096) done Nov 05 02:52:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6288(8163641) WEXITSTATUS 1 Nov 05 02:52:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6288(8163641) done Nov 05 02:52:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_237(8162042) WEXITSTATUS 0 Nov 05 02:52:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_237(8162042) done Nov 05 02:52:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_444(8162300) WEXITSTATUS 0 Nov 05 02:52:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_444(8162300) done Nov 05 02:52:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1571(8150610) WEXITSTATUS 0 Nov 05 02:52:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1571(8150610) done Nov 05 02:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5741(8162597) WEXITSTATUS 1 Nov 05 02:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5741(8162597) done Nov 05 02:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_102(8161880) WEXITSTATUS 0 Nov 05 02:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_102(8161880) done Nov 05 02:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_275(8162080) WEXITSTATUS 0 Nov 05 02:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_275(8162080) done Nov 05 02:52:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3564(8158895) WEXITSTATUS 0 Nov 05 02:52:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3564(8158895) done Nov 05 02:52:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_23(8160859) WEXITSTATUS 0 Nov 05 02:52:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_23(8160859) done Nov 05 02:52:49 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_344(8162158) WEXITSTATUS 0 Nov 05 02:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_344(8162158) done Nov 05 02:53:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_481(8162351) WEXITSTATUS 0 Nov 05 02:53:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_481(8162351) done Nov 05 02:53:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5107(8158440) WEXITSTATUS 1 Nov 05 02:53:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5107(8158440) done Nov 05 02:53:09 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:53:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:53:11 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:53:11 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:53:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:53:13 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:53:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_286(8162091) WEXITSTATUS 0 Nov 05 02:53:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_286(8162091) done Nov 05 02:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_185(8161980) WEXITSTATUS 0 Nov 05 02:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_185(8161980) done Nov 05 02:53:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_243(8162048) WEXITSTATUS 0 Nov 05 02:53:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_243(8162048) done Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_16(8159341) WEXITSTATUS 0 Nov 05 02:53:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_16(8159341) done Nov 05 02:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6158(8163205) WEXITSTATUS 1 Nov 05 02:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6158(8163205) done Nov 05 02:53:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144134 NodeList=r903u07n[03-04] #CPUs=96 Partition=scavenge Nov 05 02:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157839 WEXITSTATUS 0 Nov 05 02:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157839 done Nov 05 02:53:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_448(8162304) WEXITSTATUS 0 Nov 05 02:53:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_448(8162304) done Nov 05 02:53:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6311(8163707) WEXITSTATUS 1 Nov 05 02:53:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6311(8163707) done Nov 05 02:53:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1485(8139215) WEXITSTATUS 0 Nov 05 02:53:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1485(8139215) done Nov 05 02:53:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159923 WEXITSTATUS 0 Nov 05 02:53:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159923 done Nov 05 02:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_445(8162301) WEXITSTATUS 0 Nov 05 02:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_445(8162301) done Nov 05 02:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1423(8135956) WEXITSTATUS 0 Nov 05 02:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1423(8135956) done Nov 05 02:53:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_168(8161963) WEXITSTATUS 0 Nov 05 02:53:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_168(8161963) done Nov 05 02:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4680(8156342) WEXITSTATUS 1 Nov 05 02:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4680(8156342) done Nov 05 02:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1484(8138921) WEXITSTATUS 0 Nov 05 02:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1484(8138921) done Nov 05 02:54:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_443(8162299) WEXITSTATUS 0 Nov 05 02:54:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_443(8162299) done Nov 05 02:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_396(8162241) WEXITSTATUS 0 Nov 05 02:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_396(8162241) done Nov 05 02:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_105(8161883) WEXITSTATUS 0 Nov 05 02:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_105(8161883) done Nov 05 02:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6121(8163162) WEXITSTATUS 1 Nov 05 02:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6121(8163162) done Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_270(8163756) in pi_ohern on r917u13n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_271(8163757) in pi_ohern on r917u13n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_272(8163758) in pi_ohern on r917u13n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_273(8163759) in pi_ohern on r917u13n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_274(8163760) in pi_ohern on r917u13n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_275(8163761) in pi_ohern on r917u13n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_276(8163762) in pi_ohern on r917u13n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_277(8163763) in pi_ohern on r917u13n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_278(8163764) in pi_ohern on r917u13n03 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_279(8163765) in pi_ohern on r917u13n04 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_280(8163766) in pi_ohern on r917u29n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_281(8163767) in pi_ohern on r917u29n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_282(8163768) in pi_ohern on r917u29n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_283(8163769) in pi_ohern on r917u29n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_284(8163770) in pi_ohern on r917u29n02 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_285(8163771) in pi_ohern on r917u29n03 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_286(8163772) in pi_ohern on r917u29n03 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_287(8163773) in pi_ohern on r903u12n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_288(8163774) in pi_ohern on r911u32n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_289(8163775) in pi_ohern on r807u22n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_290(8163776) in pi_ohern on r807u22n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_291(8163777) in pi_ohern on r807u24n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_292(8163778) in pi_ohern on r807u24n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_293(8163779) in pi_ohern on r807u24n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_294(8163780) in pi_ohern on r807u24n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_295(8163781) in pi_ohern on r807u24n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_296(8163782) in pi_ohern on r913u32n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_297(8163783) in pi_ohern on r913u32n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_298(8163784) in pi_ohern on r913u35n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_299(8163785) in pi_ohern on r913u35n01 Nov 05 02:54:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162682 in day on r918u09n02 Nov 05 02:54:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3726(8163786) in gpu on r808u08n01 Nov 05 02:54:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1973(8163787) in day on r918u32n03 Nov 05 02:54:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1974(8163788) in day on r815u15n08 Nov 05 02:54:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1975(8163789) in day on r815u23n03 Nov 05 02:54:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144137 in scavenge on r915u38n[03-04] Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6326(8163790) in scavenge on r908u28n03 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6327(8163791) in scavenge on r908u37n01 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6328(8163792) in scavenge on r911u05n02 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6329(8163793) in scavenge on r815u09n11 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6330(8163794) in scavenge on r805u09n02 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6331(8163795) in scavenge on r912u32n04 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6332(8163796) in scavenge on r914u05n02 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6333(8163797) in scavenge on r909u01n03 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6334(8163798) in scavenge on r909u10n01 Nov 05 02:54:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6335(8163799) in scavenge on r909u10n01 Nov 05 02:54:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_23(8114289) in scavenge on r815u15n08 Nov 05 02:54:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_799(7572968) in scavenge on r815u23n03 Nov 05 02:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1301(8135325) WEXITSTATUS 0 Nov 05 02:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1301(8135325) done Nov 05 02:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5335(8158954) WEXITSTATUS 1 Nov 05 02:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5335(8158954) done Nov 05 02:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_87(8161865) WEXITSTATUS 0 Nov 05 02:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_87(8161865) done Nov 05 02:54:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_81(8161859) WEXITSTATUS 0 Nov 05 02:54:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_81(8161859) done Nov 05 02:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6188(8163287) WEXITSTATUS 1 Nov 05 02:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6188(8163287) done Nov 05 02:54:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144136 NodeList=r915u38n01,r917u04n01 #CPUs=96 Partition=scavenge Nov 05 02:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_480(8162350) WEXITSTATUS 0 Nov 05 02:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_480(8162350) done Nov 05 02:54:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1556(8147567) WEXITSTATUS 0 Nov 05 02:54:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1556(8147567) done Nov 05 02:54:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5092(8158416) WEXITSTATUS 1 Nov 05 02:54:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5092(8158416) done Nov 05 02:54:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159927 WEXITSTATUS 0 Nov 05 02:54:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159927 done Nov 05 02:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_78(8161856) WEXITSTATUS 0 Nov 05 02:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_78(8161856) done Nov 05 02:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1386(8135687) WEXITSTATUS 0 Nov 05 02:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1386(8135687) done Nov 05 02:54:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_160(8161955) WEXITSTATUS 0 Nov 05 02:54:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_160(8161955) done Nov 05 02:54:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_285(8162090) WEXITSTATUS 0 Nov 05 02:54:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_285(8162090) done Nov 05 02:55:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5564(8161218) WEXITSTATUS 1 Nov 05 02:55:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5564(8161218) done Nov 05 02:55:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1612(8154045) WEXITSTATUS 0 Nov 05 02:55:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1612(8154045) done Nov 05 02:55:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_86(8161864) WEXITSTATUS 0 Nov 05 02:55:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_86(8161864) done Nov 05 02:55:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_239(8162044) WEXITSTATUS 0 Nov 05 02:55:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_239(8162044) done Nov 05 02:55:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_570(8162474) WEXITSTATUS 0 Nov 05 02:55:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_570(8162474) done Nov 05 02:55:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2284(8126220) Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_456(8162326) WEXITSTATUS 0 Nov 05 02:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_456(8162326) done Nov 05 02:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5848(8162819) WEXITSTATUS 1 Nov 05 02:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5848(8162819) done Nov 05 02:55:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4822(8157801) WEXITSTATUS 1 Nov 05 02:55:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4822(8157801) done Nov 05 02:55:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_33(8161791) WEXITSTATUS 0 Nov 05 02:55:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_33(8161791) done Nov 05 02:55:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3593(8159458) WEXITSTATUS 0 Nov 05 02:55:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3593(8159458) done Nov 05 02:55:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5920(8162912) WEXITSTATUS 1 Nov 05 02:55:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5920(8162912) done Nov 05 02:55:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162683 NodeList=r806u09n03 #CPUs=16 Partition=day Nov 05 02:55:36 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 02:55:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_528(8162414) WEXITSTATUS 0 Nov 05 02:55:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_528(8162414) done Nov 05 02:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6023(8163033) WEXITSTATUS 1 Nov 05 02:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6023(8163033) done Nov 05 02:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_241(8162046) WEXITSTATUS 0 Nov 05 02:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_241(8162046) done Nov 05 02:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_183(8161978) WEXITSTATUS 0 Nov 05 02:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_183(8161978) done Nov 05 02:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_106(8161884) WEXITSTATUS 0 Nov 05 02:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_106(8161884) done Nov 05 02:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_10(8161768) WEXITSTATUS 0 Nov 05 02:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_10(8161768) done Nov 05 02:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_176(8161971) WEXITSTATUS 0 Nov 05 02:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_176(8161971) done Nov 05 02:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6226(8163407) WEXITSTATUS 1 Nov 05 02:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6226(8163407) done Nov 05 02:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6004(8163009) WEXITSTATUS 1 Nov 05 02:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6004(8163009) done Nov 05 02:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6268(8163550) WEXITSTATUS 1 Nov 05 02:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6268(8163550) done Nov 05 02:56:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6098(8163131) WEXITSTATUS 1 Nov 05 02:56:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6098(8163131) done Nov 05 02:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_482(8162352) WEXITSTATUS 0 Nov 05 02:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_482(8162352) done Nov 05 02:56:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5856(8162827) WEXITSTATUS 1 Nov 05 02:56:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5856(8162827) done Nov 05 02:56:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5987(8162987) WEXITSTATUS 1 Nov 05 02:56:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5987(8162987) done Nov 05 02:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1464(8138249) WEXITSTATUS 0 Nov 05 02:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1464(8138249) done Nov 05 02:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_107(8161885) WEXITSTATUS 0 Nov 05 02:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_107(8161885) done Nov 05 02:56:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_171(8161966) WEXITSTATUS 0 Nov 05 02:56:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_171(8161966) done Nov 05 02:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_246(8162051) WEXITSTATUS 0 Nov 05 02:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_246(8162051) done Nov 05 02:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_397(8162242) WEXITSTATUS 0 Nov 05 02:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_397(8162242) done Nov 05 02:56:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_527(8162413) WEXITSTATUS 0 Nov 05 02:56:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_527(8162413) done Nov 05 02:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_350(8162195) WEXITSTATUS 0 Nov 05 02:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_350(8162195) done Nov 05 02:56:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_395(8162240) WEXITSTATUS 0 Nov 05 02:56:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_395(8162240) done Nov 05 02:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_94(8161872) WEXITSTATUS 0 Nov 05 02:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_94(8161872) done Nov 05 02:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_457(8162327) WEXITSTATUS 0 Nov 05 02:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_457(8162327) done Nov 05 02:56:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8163800 InitPrio=50084 usec=6779 Nov 05 02:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_302(8162116) WEXITSTATUS 0 Nov 05 02:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_302(8162116) done Nov 05 02:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_77(8161855) WEXITSTATUS 0 Nov 05 02:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_77(8161855) done Nov 05 02:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_446(8162302) WEXITSTATUS 0 Nov 05 02:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_446(8162302) done Nov 05 02:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_304(8162118) WEXITSTATUS 0 Nov 05 02:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_304(8162118) done Nov 05 02:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_166(8161961) WEXITSTATUS 0 Nov 05 02:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_166(8161961) done Nov 05 02:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_25(8161035) WEXITSTATUS 0 Nov 05 02:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_25(8161035) done Nov 05 02:56:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_184(8161979) WEXITSTATUS 0 Nov 05 02:56:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_184(8161979) done Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_300(8163801) in pi_ohern on r801u32n02 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_301(8163802) in pi_ohern on r801u32n02 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_302(8163803) in pi_ohern on r917u13n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_303(8163804) in pi_ohern on r917u13n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_304(8163805) in pi_ohern on r917u13n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_305(8163806) in pi_ohern on r917u13n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_306(8163807) in pi_ohern on r917u13n02 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_307(8163808) in pi_ohern on r917u13n02 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_308(8163809) in pi_ohern on r917u13n02 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_309(8163810) in pi_ohern on r917u13n02 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_310(8163811) in pi_ohern on r917u13n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_311(8163812) in pi_ohern on r917u13n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_312(8163813) in pi_ohern on r917u13n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_313(8163814) in pi_ohern on r917u13n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_314(8163815) in pi_ohern on r917u13n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_315(8163816) in pi_ohern on r917u13n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_316(8163817) in pi_ohern on r917u13n04 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_317(8163818) in pi_ohern on r917u13n04 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_318(8163819) in pi_ohern on r917u29n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_319(8163820) in pi_ohern on r917u29n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_320(8163821) in pi_ohern on r917u29n02 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_321(8163822) in pi_ohern on r917u29n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_322(8163823) in pi_ohern on r917u29n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_323(8163824) in pi_ohern on r917u29n03 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_324(8163825) in pi_ohern on r807u22n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_325(8163826) in pi_ohern on r807u22n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_326(8163827) in pi_ohern on r807u22n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_327(8163828) in pi_ohern on r903u26n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_328(8163829) in pi_ohern on r903u26n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_329(8163830) in pi_ohern on r913u32n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_330(8163831) in pi_ohern on r913u35n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_331(8163832) in pi_ohern on r913u35n01 Nov 05 02:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_332(8163833) in pi_ohern on r913u35n01 Nov 05 02:56:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3727(8163834) in gpu on r4513u29n01 Nov 05 02:56:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1976(8163835) in day on r904u10n02 Nov 05 02:56:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1977(8163836) in day on r914u05n01 Nov 05 02:56:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1978(8163837) in day on r919u01n03 Nov 05 02:56:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1979(8163838) in day on r919u32n01 Nov 05 02:56:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 02:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_247(8162052) WEXITSTATUS 0 Nov 05 02:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_247(8162052) done Nov 05 02:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_476(8162346) WEXITSTATUS 0 Nov 05 02:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_476(8162346) done Nov 05 02:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_187(8161982) WEXITSTATUS 0 Nov 05 02:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_187(8161982) done Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6336(8163839) in scavenge on r907u01n04 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6337(8163840) in scavenge on r907u09n03 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6338(8163841) in scavenge on r802u02n03 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6339(8163842) in scavenge on r903u01n01 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6340(8163843) in scavenge on r815u09n11 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6341(8163844) in scavenge on r816u35n11 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6342(8163845) in scavenge on r807u05n01 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6343(8163846) in scavenge on r909u07n03 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6344(8163847) in scavenge on r909u10n02 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6345(8163848) in scavenge on r909u10n04 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6346(8163849) in scavenge on r909u13n01 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6347(8163850) in scavenge on r909u13n03 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6348(8163851) in scavenge on r918u32n04 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6349(8163852) in scavenge on r918u36n01 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6350(8163853) in scavenge on r918u36n01 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_886(7573055) in scavenge on r918u36n02 Nov 05 02:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1048(7573217) in scavenge on r816u35n05 Nov 05 02:56:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_159(8161954) WEXITSTATUS 0 Nov 05 02:56:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_159(8161954) done Nov 05 02:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_240(8162045) WEXITSTATUS 0 Nov 05 02:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_240(8162045) done Nov 05 02:57:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6063(8163086) WEXITSTATUS 1 Nov 05 02:57:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6063(8163086) done Nov 05 02:57:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_103(8161881) WEXITSTATUS 0 Nov 05 02:57:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_103(8161881) done Nov 05 02:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_164(8161959) WEXITSTATUS 0 Nov 05 02:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_164(8161959) done Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_524(8162410) WEXITSTATUS 0 Nov 05 02:57:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_524(8162410) done Nov 05 02:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6348(8163851) WEXITSTATUS 1 Nov 05 02:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6348(8163851) done Nov 05 02:57:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1312(8135384) WEXITSTATUS 0 Nov 05 02:57:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1312(8135384) done Nov 05 02:57:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5196(8158559) WEXITSTATUS 1 Nov 05 02:57:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5196(8158559) done Nov 05 02:57:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_526(8162412) WEXITSTATUS 0 Nov 05 02:57:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_526(8162412) done Nov 05 02:57:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_454(8162324) WEXITSTATUS 0 Nov 05 02:57:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_454(8162324) done Nov 05 02:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_191(8161986) WEXITSTATUS 0 Nov 05 02:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_191(8161986) done Nov 05 02:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5487(8160136) WEXITSTATUS 1 Nov 05 02:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5487(8160136) done Nov 05 02:57:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6142(8163183) WEXITSTATUS 1 Nov 05 02:57:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6142(8163183) done Nov 05 02:57:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163347 WEXITSTATUS 0 Nov 05 02:57:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163347 done Nov 05 02:57:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_96(8161874) WEXITSTATUS 0 Nov 05 02:57:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_96(8161874) done Nov 05 02:57:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_525(8162411) WEXITSTATUS 0 Nov 05 02:57:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_525(8162411) done Nov 05 02:58:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_186(8161981) WEXITSTATUS 0 Nov 05 02:58:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_186(8161981) done Nov 05 02:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_173(8161968) WEXITSTATUS 0 Nov 05 02:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_173(8161968) done Nov 05 02:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_315(8162129) WEXITSTATUS 0 Nov 05 02:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_315(8162129) done Nov 05 02:58:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_257(8162062) WEXITSTATUS 0 Nov 05 02:58:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_257(8162062) done Nov 05 02:58:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_569(8162473) WEXITSTATUS 0 Nov 05 02:58:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_569(8162473) done Nov 05 02:58:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 02:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_547(8162451) WEXITSTATUS 0 Nov 05 02:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_547(8162451) done Nov 05 02:58:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_303(8162117) WEXITSTATUS 0 Nov 05 02:58:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_303(8162117) done Nov 05 02:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_76(8161854) WEXITSTATUS 0 Nov 05 02:58:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_76(8161854) done Nov 05 02:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_604(8162514) WEXITSTATUS 0 Nov 05 02:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_604(8162514) done Nov 05 02:58:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_354(8162199) WEXITSTATUS 0 Nov 05 02:58:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_354(8162199) done Nov 05 02:58:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_235(8162040) WEXITSTATUS 0 Nov 05 02:58:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_235(8162040) done Nov 05 02:58:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_479(8162349) WEXITSTATUS 0 Nov 05 02:58:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_479(8162349) done Nov 05 02:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_563(8162467) WEXITSTATUS 0 Nov 05 02:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_563(8162467) done Nov 05 02:58:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_83(8161861) WEXITSTATUS 0 Nov 05 02:58:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_83(8161861) done Nov 05 02:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_358(8162203) WEXITSTATUS 0 Nov 05 02:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_358(8162203) done Nov 05 02:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_362(8162207) WEXITSTATUS 0 Nov 05 02:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_362(8162207) done Nov 05 02:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_297(8162111) WEXITSTATUS 0 Nov 05 02:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_297(8162111) done Nov 05 02:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_351(8162196) WEXITSTATUS 0 Nov 05 02:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_351(8162196) done Nov 05 02:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_355(8162200) WEXITSTATUS 0 Nov 05 02:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_355(8162200) done Nov 05 02:59:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_5(8162885) WEXITSTATUS 0 Nov 05 02:59:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_5(8162885) done Nov 05 02:59:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_458(8162328) WEXITSTATUS 0 Nov 05 02:59:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_458(8162328) done Nov 05 02:59:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_190(8161985) WEXITSTATUS 0 Nov 05 02:59:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_190(8161985) done Nov 05 02:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6175(8163261) WEXITSTATUS 1 Nov 05 02:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6175(8163261) done Nov 05 02:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5749(8162610) WEXITSTATUS 1 Nov 05 02:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5749(8162610) done Nov 05 02:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_348(8162193) WEXITSTATUS 0 Nov 05 02:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_348(8162193) done Nov 05 02:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3726(8163786) WEXITSTATUS 0 Nov 05 02:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3726(8163786) done Nov 05 02:59:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_312(8162126) WEXITSTATUS 0 Nov 05 02:59:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_312(8162126) done Nov 05 02:59:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4739(8156979) WEXITSTATUS 1 Nov 05 02:59:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4739(8156979) done Nov 05 02:59:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_299(8162113) WEXITSTATUS 0 Nov 05 02:59:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_299(8162113) done Nov 05 02:59:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_238(8162043) WEXITSTATUS 0 Nov 05 02:59:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_238(8162043) done Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 02:59:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 02:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_175(8161970) WEXITSTATUS 0 Nov 05 02:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_175(8161970) done Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_333(8163854) in pi_ohern on r801u32n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_334(8163855) in pi_ohern on r917u13n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_335(8163856) in pi_ohern on r917u13n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_336(8163857) in pi_ohern on r917u13n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_337(8163858) in pi_ohern on r917u13n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_338(8163859) in pi_ohern on r917u13n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_339(8163860) in pi_ohern on r917u13n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_340(8163861) in pi_ohern on r917u13n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_341(8163862) in pi_ohern on r917u13n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_342(8163863) in pi_ohern on r917u13n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_343(8163864) in pi_ohern on r917u13n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_344(8163865) in pi_ohern on r917u13n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_345(8163866) in pi_ohern on r917u13n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_346(8163867) in pi_ohern on r917u13n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_347(8163868) in pi_ohern on r917u13n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_348(8163869) in pi_ohern on r917u13n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_349(8163870) in pi_ohern on r917u13n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_350(8163871) in pi_ohern on r917u13n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_351(8163872) in pi_ohern on r917u13n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_352(8163873) in pi_ohern on r917u13n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_353(8163874) in pi_ohern on r917u13n04 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_354(8163875) in pi_ohern on r917u13n04 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_355(8163876) in pi_ohern on r917u29n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_356(8163877) in pi_ohern on r917u29n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_357(8163878) in pi_ohern on r917u29n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_358(8163879) in pi_ohern on r917u29n02 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_359(8163880) in pi_ohern on r917u29n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_360(8163881) in pi_ohern on r917u29n03 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_361(8163882) in pi_ohern on r917u29n04 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_362(8163883) in pi_ohern on r917u29n04 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_363(8163884) in pi_ohern on r917u29n04 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_364(8163885) in pi_ohern on r911u32n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_365(8163886) in pi_ohern on r807u22n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_366(8163887) in pi_ohern on r807u24n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_367(8163888) in pi_ohern on r807u24n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_368(8163889) in pi_ohern on r903u12n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_369(8163890) in pi_ohern on r903u26n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_370(8163891) in pi_ohern on r903u26n01 Nov 05 02:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_371(8163892) in pi_ohern on r903u26n01 Nov 05 02:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3728(8163893) in gpu on r808u08n01 Nov 05 02:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1980(8163894) in day on r918u09n04 Nov 05 02:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_349(8162194) WEXITSTATUS 0 Nov 05 02:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_349(8162194) done Nov 05 02:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6075(8163102) WEXITSTATUS 1 Nov 05 02:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6075(8163102) done Nov 05 02:59:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_571(8162475) WEXITSTATUS 0 Nov 05 02:59:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_571(8162475) done Nov 05 02:59:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_7(8158894) WEXITSTATUS 0 Nov 05 02:59:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_7(8158894) done Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6351(8163895) in scavenge on r915u35n04 Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6352(8163896) in scavenge on r815u09n10 Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6353(8163897) in scavenge on r906u39n01 Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6354(8163898) in scavenge on r806u02n04 Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6355(8163899) in scavenge on r909u13n04 Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6356(8163900) in scavenge on r918u32n04 Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6357(8163901) in scavenge on r919u01n01 Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6358(8163902) in scavenge on r919u01n01 Nov 05 02:59:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6359(8163903) in scavenge on r919u01n03 Nov 05 02:59:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_253(8162058) WEXITSTATUS 0 Nov 05 02:59:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_253(8162058) done Nov 05 02:59:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1408(8135941) WEXITSTATUS 0 Nov 05 02:59:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1408(8135941) done Nov 05 02:59:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_174(8161969) WEXITSTATUS 0 Nov 05 02:59:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_174(8161969) done Nov 05 02:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_189(8161984) WEXITSTATUS 0 Nov 05 02:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_189(8161984) done Nov 05 02:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5914(8162906) WEXITSTATUS 1 Nov 05 02:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5914(8162906) done Nov 05 02:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_562(8162466) WEXITSTATUS 0 Nov 05 02:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_562(8162466) done Nov 05 02:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_314(8162128) WEXITSTATUS 0 Nov 05 02:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_314(8162128) done Nov 05 03:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 03:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 03:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 03:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 03:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 03:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_182(8161977) WEXITSTATUS 0 Nov 05 03:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_182(8161977) done Nov 05 03:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3690(8163152) WEXITSTATUS 0 Nov 05 03:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3690(8163152) done Nov 05 03:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_170(8161965) WEXITSTATUS 0 Nov 05 03:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_170(8161965) done Nov 05 03:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_242(8162047) WEXITSTATUS 0 Nov 05 03:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_242(8162047) done Nov 05 03:00:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_319(8162133) WEXITSTATUS 0 Nov 05 03:00:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_319(8162133) done Nov 05 03:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_523(8162409) WEXITSTATUS 0 Nov 05 03:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_523(8162409) done Nov 05 03:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_167(8161962) WEXITSTATUS 0 Nov 05 03:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_167(8161962) done Nov 05 03:00:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_683(8114952) Nov 05 03:00:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6345(8163848) WEXITSTATUS 1 Nov 05 03:00:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6345(8163848) done Nov 05 03:00:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_409(8162265) WEXITSTATUS 0 Nov 05 03:00:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_409(8162265) done Nov 05 03:00:30 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8163904 NodeList=(null) usec=20174 Nov 05 03:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_529(8162415) WEXITSTATUS 0 Nov 05 03:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_529(8162415) done Nov 05 03:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5945(8162940) WEXITSTATUS 1 Nov 05 03:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5945(8162940) done Nov 05 03:00:32 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8163904 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 03:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_79(8161857) WEXITSTATUS 0 Nov 05 03:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_79(8161857) done Nov 05 03:00:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_172(8161967) WEXITSTATUS 0 Nov 05 03:00:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_172(8161967) done Nov 05 03:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_98(8161876) WEXITSTATUS 0 Nov 05 03:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_98(8161876) done Nov 05 03:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_542(8162446) WEXITSTATUS 0 Nov 05 03:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_542(8162446) done Nov 05 03:00:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_407(8162263) WEXITSTATUS 0 Nov 05 03:00:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_407(8162263) done Nov 05 03:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6055(8163078) WEXITSTATUS 1 Nov 05 03:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6055(8163078) done Nov 05 03:00:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6001(8163006) WEXITSTATUS 1 Nov 05 03:00:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6001(8163006) done Nov 05 03:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_368(8162213) WEXITSTATUS 0 Nov 05 03:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_368(8162213) done Nov 05 03:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1483(8138920) WEXITSTATUS 0 Nov 05 03:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1483(8138920) done Nov 05 03:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_49(8158572) WEXITSTATUS 0 Nov 05 03:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_49(8158572) done Nov 05 03:00:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_5(8162891) WEXITSTATUS 0 Nov 05 03:00:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_5(8162891) done Nov 05 03:00:58 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:00:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907378 WEXITSTATUS 0 Nov 05 03:00:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907378 done Nov 05 03:00:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6317(8163747) WEXITSTATUS 1 Nov 05 03:00:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6317(8163747) done Nov 05 03:00:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_572(8162476) WEXITSTATUS 0 Nov 05 03:00:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_572(8162476) done Nov 05 03:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_310(8162124) WEXITSTATUS 0 Nov 05 03:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_310(8162124) done Nov 05 03:01:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_188(8161983) WEXITSTATUS 0 Nov 05 03:01:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_188(8161983) done Nov 05 03:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_475(8162345) WEXITSTATUS 0 Nov 05 03:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_475(8162345) done Nov 05 03:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_30(8161668) WEXITSTATUS 0 Nov 05 03:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_30(8161668) done Nov 05 03:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5513(8160516) WEXITSTATUS 1 Nov 05 03:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5513(8160516) done Nov 05 03:01:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_360(8162205) WEXITSTATUS 0 Nov 05 03:01:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_360(8162205) done Nov 05 03:01:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1080(8130757) WEXITSTATUS 0 Nov 05 03:01:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1080(8130757) done Nov 05 03:01:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_406(8162262) WEXITSTATUS 0 Nov 05 03:01:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_406(8162262) done Nov 05 03:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_561(8162465) WEXITSTATUS 0 Nov 05 03:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_561(8162465) done Nov 05 03:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_256(8162061) WEXITSTATUS 0 Nov 05 03:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_256(8162061) done Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:01:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_371(8162216) WEXITSTATUS 0 Nov 05 03:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_371(8162216) done Nov 05 03:01:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_494(8162380) WEXITSTATUS 0 Nov 05 03:01:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_494(8162380) done Nov 05 03:01:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_361(8162206) WEXITSTATUS 0 Nov 05 03:01:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_361(8162206) done Nov 05 03:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_298(8162112) WEXITSTATUS 0 Nov 05 03:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_298(8162112) done Nov 05 03:01:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1492(8140411) WEXITSTATUS 0 Nov 05 03:01:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1492(8140411) done Nov 05 03:01:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_418(8162274) WEXITSTATUS 0 Nov 05 03:01:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_418(8162274) done Nov 05 03:01:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_546(8162450) WEXITSTATUS 0 Nov 05 03:01:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_546(8162450) done Nov 05 03:01:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_522(8162408) WEXITSTATUS 0 Nov 05 03:01:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_522(8162408) done Nov 05 03:01:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_230(8162035) WEXITSTATUS 0 Nov 05 03:01:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_230(8162035) done Nov 05 03:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_431(8162287) WEXITSTATUS 0 Nov 05 03:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_431(8162287) done Nov 05 03:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_376(8162221) WEXITSTATUS 0 Nov 05 03:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_376(8162221) done Nov 05 03:02:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_97(8161875) WEXITSTATUS 0 Nov 05 03:02:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_97(8161875) done Nov 05 03:02:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_248(8162053) WEXITSTATUS 0 Nov 05 03:02:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_248(8162053) done Nov 05 03:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_363(8162208) WEXITSTATUS 0 Nov 05 03:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_363(8162208) done Nov 05 03:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_245(8162050) WEXITSTATUS 0 Nov 05 03:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_245(8162050) done Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_372(8163905) in pi_ohern on r801u32n02 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_373(8163906) in pi_ohern on r801u32n02 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_374(8163907) in pi_ohern on r917u13n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_375(8163908) in pi_ohern on r917u13n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_376(8163909) in pi_ohern on r917u13n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_377(8163910) in pi_ohern on r917u13n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_378(8163911) in pi_ohern on r917u13n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_379(8163912) in pi_ohern on r917u13n02 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_380(8163913) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_381(8163914) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_382(8163915) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_383(8163916) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_384(8163917) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_385(8163918) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_386(8163919) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_387(8163920) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_388(8163921) in pi_ohern on r917u13n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_389(8163922) in pi_ohern on r917u13n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_390(8163923) in pi_ohern on r917u13n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_391(8163924) in pi_ohern on r917u13n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_392(8163925) in pi_ohern on r917u29n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_393(8163926) in pi_ohern on r917u29n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_394(8163927) in pi_ohern on r917u29n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_395(8163928) in pi_ohern on r917u29n02 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_396(8163929) in pi_ohern on r917u29n02 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_397(8163930) in pi_ohern on r917u29n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_398(8163931) in pi_ohern on r917u29n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_399(8163932) in pi_ohern on r917u29n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_400(8163933) in pi_ohern on r917u29n03 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_401(8163934) in pi_ohern on r917u29n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_402(8163935) in pi_ohern on r917u29n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_403(8163936) in pi_ohern on r917u29n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_404(8163937) in pi_ohern on r917u29n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_405(8163938) in pi_ohern on r917u29n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_406(8163939) in pi_ohern on r917u29n04 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_407(8163940) in pi_ohern on r903u26n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_408(8163941) in pi_ohern on r911u32n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_409(8163942) in pi_ohern on r807u22n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_410(8163943) in pi_ohern on r807u24n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_411(8163944) in pi_ohern on r807u24n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_412(8163945) in pi_ohern on r913u32n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_413(8163946) in pi_ohern on r913u35n01 Nov 05 03:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_414(8163947) in pi_ohern on r913u35n01 Nov 05 03:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 05 03:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n04 Nov 05 03:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907379 in gpu on r805u38n01 Nov 05 03:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3729(8163948) in gpu on r808u14n01 Nov 05 03:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1981(8163949) in day on r918u05n03 Nov 05 03:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1982(8163950) in day on r918u32n02 Nov 05 03:02:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n04 Nov 05 03:02:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8157789 Nov 05 03:02:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6360(8163951) in scavenge on r816u23n08 Nov 05 03:02:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6361(8163952) in scavenge on r909u10n04 Nov 05 03:02:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6362(8163953) in scavenge on r805u09n02 Nov 05 03:02:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6363(8163954) in scavenge on r805u14n03 Nov 05 03:02:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6364(8163955) in scavenge on r805u14n03 Nov 05 03:02:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6365(8163956) in scavenge on r806u06n02 Nov 05 03:02:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6366(8163957) in scavenge on r806u09n04 Nov 05 03:02:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_541(7572710) in scavenge on r805u38n01 Nov 05 03:02:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162639 WEXITSTATUS 0 Nov 05 03:02:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162639 done Nov 05 03:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_412(8162268) WEXITSTATUS 0 Nov 05 03:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_412(8162268) done Nov 05 03:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 03:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 03:02:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 03:02:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1042(8128646) WEXITSTATUS 0 Nov 05 03:02:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1042(8128646) done Nov 05 03:02:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_258(8162063) WEXITSTATUS 0 Nov 05 03:02:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_258(8162063) done Nov 05 03:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_356(8162201) WEXITSTATUS 0 Nov 05 03:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_356(8162201) done Nov 05 03:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_353(8162198) WEXITSTATUS 0 Nov 05 03:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_353(8162198) done Nov 05 03:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3728(8163893) WEXITSTATUS 0 Nov 05 03:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3728(8163893) done Nov 05 03:02:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_311(8162125) WEXITSTATUS 0 Nov 05 03:02:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_311(8162125) done Nov 05 03:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5962(8162962) WEXITSTATUS 1 Nov 05 03:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5962(8162962) done Nov 05 03:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6277(8163596) WEXITSTATUS 1 Nov 05 03:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6277(8163596) done Nov 05 03:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_564(8162468) WEXITSTATUS 0 Nov 05 03:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_564(8162468) done Nov 05 03:03:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_109(8161887) WEXITSTATUS 0 Nov 05 03:03:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_109(8161887) done Nov 05 03:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_255(8162060) WEXITSTATUS 0 Nov 05 03:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_255(8162060) done Nov 05 03:03:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_318(8162132) WEXITSTATUS 0 Nov 05 03:03:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_318(8162132) done Nov 05 03:03:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6325(8163755) WEXITSTATUS 1 Nov 05 03:03:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6325(8163755) done Nov 05 03:03:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8157792: QOS part_gpu_devel TRES gres/gpu:a5000 grp_used_tres_run_secs underflow, tried to remove 47 seconds when only 11 remained. Nov 05 03:03:18 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8157792: QOS part_gpu_devel TRES gres/gpu:a5000 grp_used_tres_run_secs underflow, tried to remove 47 seconds when only 11 remained. Nov 05 03:03:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8157792 Nov 05 03:03:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_370(8162215) WEXITSTATUS 0 Nov 05 03:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_370(8162215) done Nov 05 03:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4323(8154387) WEXITSTATUS 1 Nov 05 03:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4323(8154387) done Nov 05 03:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1361(8135624) WEXITSTATUS 0 Nov 05 03:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1361(8135624) done Nov 05 03:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_307(8162121) WEXITSTATUS 0 Nov 05 03:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_307(8162121) done Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:03:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:03:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5860(8162831) WEXITSTATUS 1 Nov 05 03:03:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5860(8162831) done Nov 05 03:03:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_413(8162269) WEXITSTATUS 0 Nov 05 03:03:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_413(8162269) done Nov 05 03:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_498(8162384) WEXITSTATUS 0 Nov 05 03:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_498(8162384) done Nov 05 03:03:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_497(8162383) WEXITSTATUS 0 Nov 05 03:03:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_497(8162383) done Nov 05 03:03:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_606(8162516) WEXITSTATUS 0 Nov 05 03:03:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_606(8162516) done Nov 05 03:03:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_359(8162204) WEXITSTATUS 0 Nov 05 03:03:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_359(8162204) done Nov 05 03:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_155(8161950) WEXITSTATUS 0 Nov 05 03:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_155(8161950) done Nov 05 03:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_366(8162211) WEXITSTATUS 0 Nov 05 03:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_366(8162211) done Nov 05 03:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_411(8162267) WEXITSTATUS 0 Nov 05 03:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_411(8162267) done Nov 05 03:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_367(8162212) WEXITSTATUS 0 Nov 05 03:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_367(8162212) done Nov 05 03:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_544(8162448) WEXITSTATUS 0 Nov 05 03:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_544(8162448) done Nov 05 03:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_249(8162054) WEXITSTATUS 0 Nov 05 03:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_249(8162054) done Nov 05 03:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_565(8162469) WEXITSTATUS 0 Nov 05 03:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_565(8162469) done Nov 05 03:04:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6116(8163157) WEXITSTATUS 1 Nov 05 03:04:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6116(8163157) done Nov 05 03:04:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1260(8135165) WEXITSTATUS 0 Nov 05 03:04:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1260(8135165) done Nov 05 03:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_410(8162266) WEXITSTATUS 0 Nov 05 03:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_410(8162266) done Nov 05 03:04:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8157816 Nov 05 03:04:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_405(8162261) WEXITSTATUS 0 Nov 05 03:04:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_405(8162261) done Nov 05 03:04:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_543(8162447) WEXITSTATUS 0 Nov 05 03:04:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_543(8162447) done Nov 05 03:04:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_549(8162453) WEXITSTATUS 0 Nov 05 03:04:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_549(8162453) done Nov 05 03:04:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_227(8162032) WEXITSTATUS 0 Nov 05 03:04:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_227(8162032) done Nov 05 03:04:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 03:04:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 03:04:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 03:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_373(8162218) WEXITSTATUS 0 Nov 05 03:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_373(8162218) done Nov 05 03:04:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_452(8162322) WEXITSTATUS 0 Nov 05 03:04:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_452(8162322) done Nov 05 03:04:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5521(8160700) WEXITSTATUS 1 Nov 05 03:04:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5521(8160700) done Nov 05 03:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_352(8162197) WEXITSTATUS 0 Nov 05 03:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_352(8162197) done Nov 05 03:04:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1405(8135938) WEXITSTATUS 0 Nov 05 03:04:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1405(8135938) done Nov 05 03:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_401(8162257) WEXITSTATUS 0 Nov 05 03:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_401(8162257) done Nov 05 03:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_414(8162270) WEXITSTATUS 0 Nov 05 03:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_414(8162270) done Nov 05 03:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_300(8162114) WEXITSTATUS 0 Nov 05 03:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_300(8162114) done Nov 05 03:04:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_415(8163958) in pi_ohern on r801u32n02 Nov 05 03:04:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_416(8163959) in pi_ohern on r801u32n02 Nov 05 03:04:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_417(8163960) in pi_ohern on r917u13n01 Nov 05 03:04:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_418(8163961) in pi_ohern on r917u13n01 Nov 05 03:04:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_419(8163962) in pi_ohern on r917u13n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_420(8163963) in pi_ohern on r917u13n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_421(8163964) in pi_ohern on r917u13n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_422(8163965) in pi_ohern on r917u13n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_423(8163966) in pi_ohern on r917u13n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_424(8163967) in pi_ohern on r917u13n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_425(8163968) in pi_ohern on r917u13n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_426(8163969) in pi_ohern on r917u13n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_427(8163970) in pi_ohern on r917u13n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_428(8163971) in pi_ohern on r917u13n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_429(8163972) in pi_ohern on r917u13n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_430(8163973) in pi_ohern on r917u13n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_431(8163974) in pi_ohern on r917u13n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_432(8163975) in pi_ohern on r917u13n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_433(8163976) in pi_ohern on r917u13n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_434(8163977) in pi_ohern on r917u29n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_435(8163978) in pi_ohern on r917u29n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_436(8163979) in pi_ohern on r917u29n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_437(8163980) in pi_ohern on r917u29n02 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_438(8163981) in pi_ohern on r917u29n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_439(8163982) in pi_ohern on r917u29n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_440(8163983) in pi_ohern on r917u29n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_441(8163984) in pi_ohern on r917u29n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_442(8163985) in pi_ohern on r917u29n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_443(8163986) in pi_ohern on r917u29n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_444(8163987) in pi_ohern on r917u29n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_445(8163988) in pi_ohern on r917u29n04 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_446(8163989) in pi_ohern on r903u12n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_447(8163990) in pi_ohern on r903u12n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_448(8163991) in pi_ohern on r911u32n01 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162686 in day on r815u03n03 Nov 05 03:04:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3730(8163992) in gpu on r808u08n01 Nov 05 03:05:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1983(8163993) in day on r912u28n04 Nov 05 03:05:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1984(8163994) in day on r918u09n04 Nov 05 03:05:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_420(8162276) WEXITSTATUS 0 Nov 05 03:05:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_420(8162276) done Nov 05 03:05:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6367(8163995) in scavenge on r908u35n04 Nov 05 03:05:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6368(8163996) in scavenge on r801u04n01 Nov 05 03:05:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6369(8163997) in scavenge on r802u09n01 Nov 05 03:05:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6370(8163998) in scavenge on r815u15n11 Nov 05 03:05:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6371(8163999) in scavenge on r816u15n12 Nov 05 03:05:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6372(8164000) in scavenge on r909u07n04 Nov 05 03:05:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6373(8164001) in scavenge on r918u01n03 Nov 05 03:05:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3527(8135673) WEXITSTATUS 1 Nov 05 03:05:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3527(8135673) done Nov 05 03:05:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_429(8162285) WEXITSTATUS 0 Nov 05 03:05:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_429(8162285) done Nov 05 03:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3708(8163303) WEXITSTATUS 0 Nov 05 03:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3708(8163303) done Nov 05 03:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_250(8162055) WEXITSTATUS 0 Nov 05 03:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_250(8162055) done Nov 05 03:05:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_580(8162490) WEXITSTATUS 0 Nov 05 03:05:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_580(8162490) done Nov 05 03:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_462(8162332) WEXITSTATUS 0 Nov 05 03:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_462(8162332) done Nov 05 03:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5879(8162855) WEXITSTATUS 1 Nov 05 03:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5879(8162855) done Nov 05 03:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_95(8161873) WEXITSTATUS 0 Nov 05 03:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_95(8161873) done Nov 05 03:05:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8157822 Nov 05 03:05:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_495(8162381) WEXITSTATUS 0 Nov 05 03:05:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_495(8162381) done Nov 05 03:05:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_451(8162321) WEXITSTATUS 0 Nov 05 03:05:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_451(8162321) done Nov 05 03:05:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1274(8135229) WEXITSTATUS 0 Nov 05 03:05:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1274(8135229) done Nov 05 03:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_251(8162056) WEXITSTATUS 0 Nov 05 03:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_251(8162056) done Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_404(8162260) WEXITSTATUS 0 Nov 05 03:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_404(8162260) done Nov 05 03:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1407(8135940) WEXITSTATUS 0 Nov 05 03:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1407(8135940) done Nov 05 03:05:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1439(8135972) WEXITSTATUS 0 Nov 05 03:05:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1439(8135972) done Nov 05 03:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8163800 NodeList=r815u29n06 #CPUs=10 Partition=day Nov 05 03:05:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_566(8162470) WEXITSTATUS 0 Nov 05 03:05:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_566(8162470) done Nov 05 03:05:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_228(8162033) WEXITSTATUS 0 Nov 05 03:05:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_228(8162033) done Nov 05 03:05:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_377(8162222) WEXITSTATUS 0 Nov 05 03:05:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_377(8162222) done Nov 05 03:05:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_369(8162214) WEXITSTATUS 0 Nov 05 03:05:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_369(8162214) done Nov 05 03:05:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_430(8162286) WEXITSTATUS 0 Nov 05 03:05:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_430(8162286) done Nov 05 03:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_317(8162131) WEXITSTATUS 0 Nov 05 03:05:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_317(8162131) done Nov 05 03:05:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_567(8162471) WEXITSTATUS 0 Nov 05 03:05:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_567(8162471) done Nov 05 03:05:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_605(8162515) WEXITSTATUS 0 Nov 05 03:05:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_605(8162515) done Nov 05 03:06:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_372(8162217) WEXITSTATUS 0 Nov 05 03:06:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_372(8162217) done Nov 05 03:06:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3723(8163637) WEXITSTATUS 0 Nov 05 03:06:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3723(8163637) done Nov 05 03:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_486(8162372) WEXITSTATUS 0 Nov 05 03:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_486(8162372) done Nov 05 03:06:13 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:06:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:06:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:06:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:06:16 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:06:17 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:06:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:06:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5537(8160872) WEXITSTATUS 1 Nov 05 03:06:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5537(8160872) done Nov 05 03:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_374(8162219) WEXITSTATUS 0 Nov 05 03:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_374(8162219) done Nov 05 03:06:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_301(8162115) WEXITSTATUS 0 Nov 05 03:06:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_301(8162115) done Nov 05 03:06:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_402(8162258) WEXITSTATUS 0 Nov 05 03:06:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_402(8162258) done Nov 05 03:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6342(8163845) WEXITSTATUS 1 Nov 05 03:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6342(8163845) done Nov 05 03:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_496(8162382) WEXITSTATUS 0 Nov 05 03:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_496(8162382) done Nov 05 03:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5413(8159394) WEXITSTATUS 1 Nov 05 03:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5413(8159394) done Nov 05 03:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_568(8162472) WEXITSTATUS 0 Nov 05 03:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_568(8162472) done Nov 05 03:06:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_403(8162259) WEXITSTATUS 0 Nov 05 03:06:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_403(8162259) done Nov 05 03:06:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_417(8162273) WEXITSTATUS 0 Nov 05 03:06:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_417(8162273) done Nov 05 03:06:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_538(8162442) WEXITSTATUS 0 Nov 05 03:06:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_538(8162442) done Nov 05 03:06:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_309(8162123) WEXITSTATUS 0 Nov 05 03:06:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_309(8162123) done Nov 05 03:06:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_545(8162449) WEXITSTATUS 0 Nov 05 03:06:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_545(8162449) done Nov 05 03:06:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6244(8163450) WEXITSTATUS 1 Nov 05 03:06:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6244(8163450) done Nov 05 03:06:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_308(8162122) WEXITSTATUS 0 Nov 05 03:06:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_308(8162122) done Nov 05 03:06:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_9(8159013) WEXITSTATUS 0 Nov 05 03:06:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_9(8159013) done Nov 05 03:07:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_483(8162369) WEXITSTATUS 0 Nov 05 03:07:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_483(8162369) done Nov 05 03:07:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_42(8158712) WEXITSTATUS 0 Nov 05 03:07:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158572_42(8158712) done Nov 05 03:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_293(8162107) WEXITSTATUS 0 Nov 05 03:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_293(8162107) done Nov 05 03:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_432(8162288) WEXITSTATUS 0 Nov 05 03:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_432(8162288) done Nov 05 03:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_548(8162452) WEXITSTATUS 0 Nov 05 03:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_548(8162452) done Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:07:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_232(8162037) WEXITSTATUS 0 Nov 05 03:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_232(8162037) done Nov 05 03:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_469(8162339) WEXITSTATUS 0 Nov 05 03:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_469(8162339) done Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_449(8164002) in pi_ohern on r801u32n02 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_450(8164003) in pi_ohern on r801u32n02 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_451(8164004) in pi_ohern on r801u32n02 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_452(8164005) in pi_ohern on r917u13n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_453(8164006) in pi_ohern on r917u13n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_454(8164007) in pi_ohern on r917u13n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_455(8164008) in pi_ohern on r917u13n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_456(8164009) in pi_ohern on r917u13n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_457(8164010) in pi_ohern on r917u13n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_458(8164011) in pi_ohern on r917u13n02 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_459(8164012) in pi_ohern on r917u13n03 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_460(8164013) in pi_ohern on r917u13n03 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_461(8164014) in pi_ohern on r917u13n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_462(8164015) in pi_ohern on r917u13n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_463(8164016) in pi_ohern on r917u13n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_464(8164017) in pi_ohern on r917u13n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_465(8164018) in pi_ohern on r917u13n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_466(8164019) in pi_ohern on r917u29n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_467(8164020) in pi_ohern on r917u29n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_468(8164021) in pi_ohern on r917u29n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_469(8164022) in pi_ohern on r917u29n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_470(8164023) in pi_ohern on r917u29n02 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_471(8164024) in pi_ohern on r917u29n02 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_472(8164025) in pi_ohern on r917u29n03 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_473(8164026) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_474(8164027) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_475(8164028) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_476(8164029) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_477(8164030) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_478(8164031) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_479(8164032) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_480(8164033) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_481(8164034) in pi_ohern on r917u29n04 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_482(8164035) in pi_ohern on r903u12n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_483(8164036) in pi_ohern on r903u12n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_484(8164037) in pi_ohern on r903u12n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_485(8164038) in pi_ohern on r903u26n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3731(8164039) in gpu on r808u08n01 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_416(8162272) WEXITSTATUS 0 Nov 05 03:07:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_416(8162272) done Nov 05 03:07:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3732(8164040) in gpu on r808u17n01 Nov 05 03:07:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1985(8164041) in day on r914u01n04 Nov 05 03:07:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1986(8164042) in day on r815u15n04 Nov 05 03:07:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1987(8164043) in day on r815u15n07 Nov 05 03:07:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1988(8164044) in day on r815u15n07 Nov 05 03:07:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_424(8162280) WEXITSTATUS 0 Nov 05 03:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_424(8162280) done Nov 05 03:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1514(8143681) WEXITSTATUS 0 Nov 05 03:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1514(8143681) done Nov 05 03:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159931 WEXITSTATUS 0 Nov 05 03:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159931 done Nov 05 03:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3729(8163948) WEXITSTATUS 0 Nov 05 03:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3729(8163948) done Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6374(8164045) in scavenge on r802u11n03 Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6375(8164046) in scavenge on r815u09n12 Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6376(8164047) in scavenge on r816u23n12 Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6377(8164048) in scavenge on r816u29n11 Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6378(8164049) in scavenge on r806u04n04 Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6379(8164050) in scavenge on r806u04n04 Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_736(7572905) in scavenge on r913u24n01 Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1155(7573324) in scavenge on r919u36n02 Nov 05 03:07:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1160(7573329) in scavenge on r815u15n04 Nov 05 03:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_453(8162323) WEXITSTATUS 0 Nov 05 03:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_453(8162323) done Nov 05 03:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_455(8162325) WEXITSTATUS 0 Nov 05 03:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_455(8162325) done Nov 05 03:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_472(8162342) WEXITSTATUS 0 Nov 05 03:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_472(8162342) done Nov 05 03:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_408(8162264) WEXITSTATUS 0 Nov 05 03:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_408(8162264) done Nov 05 03:07:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_535(8162439) WEXITSTATUS 0 Nov 05 03:07:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_535(8162439) done Nov 05 03:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_489(8162375) WEXITSTATUS 0 Nov 05 03:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_489(8162375) done Nov 05 03:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1410(8135943) WEXITSTATUS 0 Nov 05 03:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1410(8135943) done Nov 05 03:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_470(8162340) WEXITSTATUS 0 Nov 05 03:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_470(8162340) done Nov 05 03:08:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_427(8162283) WEXITSTATUS 0 Nov 05 03:08:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_427(8162283) done Nov 05 03:08:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_493(8162379) WEXITSTATUS 0 Nov 05 03:08:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_493(8162379) done Nov 05 03:08:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_603(8162513) WEXITSTATUS 0 Nov 05 03:08:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_603(8162513) done Nov 05 03:08:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_6(8162879) WEXITSTATUS 0 Nov 05 03:08:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162879_6(8162879) done Nov 05 03:08:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162638 WEXITSTATUS 0 Nov 05 03:08:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162638 done Nov 05 03:08:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6299(8163652) WEXITSTATUS 1 Nov 05 03:08:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6299(8163652) done Nov 05 03:08:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_640(7572809) Nov 05 03:08:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5851(8162822) WEXITSTATUS 1 Nov 05 03:08:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5851(8162822) done Nov 05 03:08:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_365(8162210) WEXITSTATUS 0 Nov 05 03:08:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_365(8162210) done Nov 05 03:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1280(8135259) WEXITSTATUS 0 Nov 05 03:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1280(8135259) done Nov 05 03:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_607(8162517) WEXITSTATUS 0 Nov 05 03:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_607(8162517) done Nov 05 03:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_357(8162202) WEXITSTATUS 0 Nov 05 03:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_357(8162202) done Nov 05 03:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_415(8162271) WEXITSTATUS 0 Nov 05 03:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_415(8162271) done Nov 05 03:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162684 NodeList=r815u29n03 #CPUs=16 Partition=day Nov 05 03:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162685 NodeList=r815u35n06 #CPUs=16 Partition=day Nov 05 03:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_581(8162491) WEXITSTATUS 0 Nov 05 03:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_581(8162491) done Nov 05 03:08:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5835(8162802) WEXITSTATUS 1 Nov 05 03:08:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5835(8162802) done Nov 05 03:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_347(8162192) WEXITSTATUS 0 Nov 05 03:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_347(8162192) done Nov 05 03:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159928 WEXITSTATUS 0 Nov 05 03:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159928 done Nov 05 03:08:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3677(8163057) WEXITSTATUS 0 Nov 05 03:08:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3677(8163057) done Nov 05 03:08:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:08:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:08:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:08:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6358(8163902) WEXITSTATUS 1 Nov 05 03:08:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6358(8163902) done Nov 05 03:08:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159935 WEXITSTATUS 0 Nov 05 03:08:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159935 done Nov 05 03:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_464(8162334) WEXITSTATUS 0 Nov 05 03:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_464(8162334) done Nov 05 03:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6357(8163901) WEXITSTATUS 1 Nov 05 03:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6357(8163901) done Nov 05 03:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159938 WEXITSTATUS 0 Nov 05 03:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159938 done Nov 05 03:08:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_506(8162392) WEXITSTATUS 0 Nov 05 03:08:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_506(8162392) done Nov 05 03:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_485(8162371) WEXITSTATUS 0 Nov 05 03:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_485(8162371) done Nov 05 03:09:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6129(8163170) WEXITSTATUS 1 Nov 05 03:09:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6129(8163170) done Nov 05 03:09:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_501(8162387) WEXITSTATUS 0 Nov 05 03:09:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_501(8162387) done Nov 05 03:09:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_254(8162059) WEXITSTATUS 0 Nov 05 03:09:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_254(8162059) done Nov 05 03:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_428(8162284) WEXITSTATUS 0 Nov 05 03:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_428(8162284) done Nov 05 03:09:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_229(8162034) WEXITSTATUS 0 Nov 05 03:09:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_229(8162034) done Nov 05 03:09:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6143(8163184) WEXITSTATUS 1 Nov 05 03:09:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6143(8163184) done Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5875(8162847) WEXITSTATUS 1 Nov 05 03:09:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5875(8162847) done Nov 05 03:09:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162687 NodeList=r815u29n02 #CPUs=16 Partition=day Nov 05 03:09:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162688 NodeList=r816u03n03 #CPUs=16 Partition=day Nov 05 03:09:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162689 NodeList=r816u03n08 #CPUs=16 Partition=day Nov 05 03:09:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_490(8162376) WEXITSTATUS 0 Nov 05 03:09:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_490(8162376) done Nov 05 03:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_425(8162281) WEXITSTATUS 0 Nov 05 03:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_425(8162281) done Nov 05 03:09:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_419(8162275) WEXITSTATUS 0 Nov 05 03:09:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_419(8162275) done Nov 05 03:09:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5687(8162309) WEXITSTATUS 1 Nov 05 03:09:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5687(8162309) done Nov 05 03:09:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_507(8162393) WEXITSTATUS 0 Nov 05 03:09:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_507(8162393) done Nov 05 03:09:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1608(8153647) WEXITSTATUS 0 Nov 05 03:09:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1608(8153647) done Nov 05 03:09:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3918(8145427) WEXITSTATUS 1 Nov 05 03:09:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3918(8145427) done Nov 05 03:09:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1338(8135476) WEXITSTATUS 0 Nov 05 03:09:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1338(8135476) done Nov 05 03:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_533(8162437) WEXITSTATUS 0 Nov 05 03:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_533(8162437) done Nov 05 03:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_450(8162320) WEXITSTATUS 0 Nov 05 03:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_450(8162320) done Nov 05 03:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4623(8156207) WEXITSTATUS 1 Nov 05 03:09:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4623(8156207) done Nov 05 03:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_492(8162378) WEXITSTATUS 0 Nov 05 03:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_492(8162378) done Nov 05 03:10:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_465(8162335) WEXITSTATUS 0 Nov 05 03:10:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_465(8162335) done Nov 05 03:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_252(8162057) WEXITSTATUS 0 Nov 05 03:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_252(8162057) done Nov 05 03:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_463(8162333) WEXITSTATUS 0 Nov 05 03:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_463(8162333) done Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_486(8164051) in pi_ohern on r801u32n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_487(8164052) in pi_ohern on r801u32n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_488(8164053) in pi_ohern on r917u13n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_489(8164054) in pi_ohern on r917u13n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_490(8164055) in pi_ohern on r917u13n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_491(8164056) in pi_ohern on r917u13n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_492(8164057) in pi_ohern on r917u13n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_493(8164058) in pi_ohern on r917u13n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_494(8164059) in pi_ohern on r917u13n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_495(8164060) in pi_ohern on r917u13n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_496(8164061) in pi_ohern on r917u13n03 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_497(8164062) in pi_ohern on r917u13n03 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_498(8164063) in pi_ohern on r917u13n03 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_499(8164064) in pi_ohern on r917u13n03 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_500(8164065) in pi_ohern on r917u13n04 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_501(8164066) in pi_ohern on r917u29n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_502(8164067) in pi_ohern on r917u29n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_503(8164068) in pi_ohern on r917u29n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_504(8164069) in pi_ohern on r917u29n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_505(8164070) in pi_ohern on r917u29n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_506(8164071) in pi_ohern on r917u29n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_507(8164072) in pi_ohern on r917u29n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_508(8164073) in pi_ohern on r917u29n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_509(8164074) in pi_ohern on r917u29n02 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_510(8164075) in pi_ohern on r917u29n03 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_511(8164076) in pi_ohern on r917u29n03 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_512(8164077) in pi_ohern on r917u29n03 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_513(8164078) in pi_ohern on r917u29n03 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_514(8164079) in pi_ohern on r917u29n04 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_515(8164080) in pi_ohern on r917u29n04 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_516(8164081) in pi_ohern on r917u29n04 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_517(8164082) in pi_ohern on r917u29n04 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_518(8164083) in pi_ohern on r917u29n04 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_519(8164084) in pi_ohern on r807u24n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_520(8164085) in pi_ohern on r913u35n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3733(8164086) in gpu on r808u14n01 Nov 05 03:10:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3734(8164087) in gpu on r4513u23n01 Nov 05 03:10:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1989(8164088) in day on r909u07n04 Nov 05 03:10:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1990(8164089) in day on r909u10n01 Nov 05 03:10:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1991(8164090) in day on r815u35n02 Nov 05 03:10:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_461(8162331) WEXITSTATUS 0 Nov 05 03:10:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_461(8162331) done Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_531(8162435) WEXITSTATUS 0 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_531(8162435) done Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6380(8164091) in scavenge on r801u09n03 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6381(8164092) in scavenge on r802u30n01 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6382(8164093) in scavenge on r807u05n01 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6383(8164094) in scavenge on r909u13n04 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6384(8164095) in scavenge on r904u04n04 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6385(8164096) in scavenge on r904u10n01 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6386(8164097) in scavenge on r904u10n01 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6387(8164098) in scavenge on r912u32n03 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6388(8164099) in scavenge on r913u24n01 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6389(8164100) in scavenge on r913u28n01 Nov 05 03:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6390(8164101) in scavenge on r918u05n03 Nov 05 03:10:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1596(8152594) WEXITSTATUS 0 Nov 05 03:10:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1596(8152594) done Nov 05 03:10:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_537(8162441) WEXITSTATUS 0 Nov 05 03:10:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_537(8162441) done Nov 05 03:10:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_6(8162878) WEXITSTATUS 0 Nov 05 03:10:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162878_6(8162878) done Nov 05 03:10:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_313(8162127) WEXITSTATUS 0 Nov 05 03:10:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_313(8162127) done Nov 05 03:10:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_231(8162036) WEXITSTATUS 0 Nov 05 03:10:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_231(8162036) done Nov 05 03:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_459(8162329) WEXITSTATUS 0 Nov 05 03:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_459(8162329) done Nov 05 03:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2416(8152460) WEXITSTATUS 0 Nov 05 03:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2416(8152460) done Nov 05 03:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_468(8162338) WEXITSTATUS 0 Nov 05 03:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_468(8162338) done Nov 05 03:10:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7569662_1195(7570857) Nov 05 03:10:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8157864 Nov 05 03:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5974(8162974) WEXITSTATUS 1 Nov 05 03:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5974(8162974) done Nov 05 03:10:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_532(8162436) WEXITSTATUS 0 Nov 05 03:10:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_532(8162436) done Nov 05 03:10:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_292(8162106) WEXITSTATUS 0 Nov 05 03:10:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_292(8162106) done Nov 05 03:10:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6194(8163316) WEXITSTATUS 1 Nov 05 03:10:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6194(8163316) done Nov 05 03:11:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3731(8164039) WEXITSTATUS 0 Nov 05 03:11:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3731(8164039) done Nov 05 03:11:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_484(8162370) WEXITSTATUS 0 Nov 05 03:11:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_484(8162370) done Nov 05 03:11:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159225 WEXITSTATUS 0 Nov 05 03:11:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159225 done Nov 05 03:11:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5889(8162865) WEXITSTATUS 1 Nov 05 03:11:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5889(8162865) done Nov 05 03:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_471(8162341) WEXITSTATUS 0 Nov 05 03:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_471(8162341) done Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_487(8162373) WEXITSTATUS 0 Nov 05 03:11:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_487(8162373) done Nov 05 03:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6029(8163045) WEXITSTATUS 1 Nov 05 03:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6029(8163045) done Nov 05 03:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_14(8159259) WEXITSTATUS 0 Nov 05 03:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_14(8159259) done Nov 05 03:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6383(8164094) WEXITSTATUS 1 Nov 05 03:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6383(8164094) done Nov 05 03:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6125(8163166) WEXITSTATUS 1 Nov 05 03:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6125(8163166) done Nov 05 03:11:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_578(8162488) WEXITSTATUS 0 Nov 05 03:11:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_578(8162488) done Nov 05 03:11:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_449(8162319) WEXITSTATUS 0 Nov 05 03:11:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_449(8162319) done Nov 05 03:11:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5684(8162306) WEXITSTATUS 1 Nov 05 03:11:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5684(8162306) done Nov 05 03:11:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_375(8162220) WEXITSTATUS 0 Nov 05 03:11:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_375(8162220) done Nov 05 03:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_509(8162395) WEXITSTATUS 0 Nov 05 03:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_509(8162395) done Nov 05 03:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_502(8162388) WEXITSTATUS 0 Nov 05 03:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_502(8162388) done Nov 05 03:11:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1546(8146915) WEXITSTATUS 0 Nov 05 03:11:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1546(8146915) done Nov 05 03:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5696(8162358) WEXITSTATUS 1 Nov 05 03:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5696(8162358) done Nov 05 03:12:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6117(8163158) WEXITSTATUS 1 Nov 05 03:12:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6117(8163158) done Nov 05 03:12:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6233(8163414) WEXITSTATUS 1 Nov 05 03:12:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6233(8163414) done Nov 05 03:12:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_536(8162440) WEXITSTATUS 0 Nov 05 03:12:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_536(8162440) done Nov 05 03:12:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_316(8162130) WEXITSTATUS 0 Nov 05 03:12:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_316(8162130) done Nov 05 03:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_541(8162445) WEXITSTATUS 0 Nov 05 03:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_541(8162445) done Nov 05 03:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_491(8162377) WEXITSTATUS 0 Nov 05 03:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_491(8162377) done Nov 05 03:12:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6155(8163202) WEXITSTATUS 1 Nov 05 03:12:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6155(8163202) done Nov 05 03:12:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1623(8154569) WEXITSTATUS 0 Nov 05 03:12:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1623(8154569) done Nov 05 03:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_467(8162337) WEXITSTATUS 0 Nov 05 03:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_467(8162337) done Nov 05 03:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_488(8162374) WEXITSTATUS 0 Nov 05 03:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_488(8162374) done Nov 05 03:12:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_421(8162277) WEXITSTATUS 0 Nov 05 03:12:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_421(8162277) done Nov 05 03:12:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_576(8162486) WEXITSTATUS 0 Nov 05 03:12:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_576(8162486) done Nov 05 03:12:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_294(8162108) WEXITSTATUS 0 Nov 05 03:12:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_294(8162108) done Nov 05 03:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133382 WEXITSTATUS 0 Nov 05 03:12:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133382 done Nov 05 03:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2563(8152830) WEXITSTATUS 0 Nov 05 03:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2563(8152830) done Nov 05 03:12:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3721(8163534) WEXITSTATUS 0 Nov 05 03:12:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3721(8163534) done Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_521(8164102) in pi_ohern on r801u32n02 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_522(8164103) in pi_ohern on r801u32n02 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_523(8164104) in pi_ohern on r801u32n02 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_524(8164105) in pi_ohern on r917u13n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_525(8164106) in pi_ohern on r917u13n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_526(8164107) in pi_ohern on r917u13n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_527(8164108) in pi_ohern on r917u13n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_528(8164109) in pi_ohern on r917u13n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_529(8164110) in pi_ohern on r917u13n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_530(8164111) in pi_ohern on r917u13n02 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_531(8164112) in pi_ohern on r917u13n02 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_532(8164113) in pi_ohern on r917u13n02 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_533(8164114) in pi_ohern on r917u13n03 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_534(8164115) in pi_ohern on r917u13n03 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_535(8164116) in pi_ohern on r917u13n03 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_536(8164117) in pi_ohern on r917u29n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_537(8164118) in pi_ohern on r917u29n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_538(8164119) in pi_ohern on r917u29n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_539(8164120) in pi_ohern on r917u29n02 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_540(8164121) in pi_ohern on r917u29n02 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_541(8164122) in pi_ohern on r917u29n03 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_542(8164123) in pi_ohern on r917u29n03 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_543(8164124) in pi_ohern on r917u29n03 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_544(8164125) in pi_ohern on r917u29n04 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_545(8164126) in pi_ohern on r917u29n04 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_546(8164127) in pi_ohern on r917u29n04 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3735(8164128) in gpu on r808u08n01 Nov 05 03:12:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3736(8164129) in gpu on r808u08n01 Nov 05 03:13:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1992(8164130) in day on r815u23n03 Nov 05 03:13:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1993(8164131) in day on r816u03n04 Nov 05 03:13:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133397 in gpu on r908u24n01 Nov 05 03:13:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_550(8162454) WEXITSTATUS 0 Nov 05 03:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_550(8162454) done Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6391(8164132) in scavenge on r908u37n04 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6392(8164133) in scavenge on r801u06n02 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6393(8164134) in scavenge on r808u08n01 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6394(8164135) in scavenge on r903u34n01 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6395(8164136) in scavenge on r807u11n01 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6396(8164137) in scavenge on r909u13n04 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6397(8164138) in scavenge on r805u02n01 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6398(8164139) in scavenge on r805u23n04 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6399(8164140) in scavenge on r805u23n04 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6400(8164141) in scavenge on r912u28n04 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6401(8164142) in scavenge on r913u01n01 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_299(8114565) in scavenge on r908u35n01 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_558(7572727) in scavenge on r802u11n01 Nov 05 03:13:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_28(8114294) in scavenge on r913u13n03 Nov 05 03:13:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159223 WEXITSTATUS 0 Nov 05 03:13:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159223 done Nov 05 03:13:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5873(8162845) WEXITSTATUS 1 Nov 05 03:13:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5873(8162845) done Nov 05 03:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_556(8162460) WEXITSTATUS 0 Nov 05 03:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_556(8162460) done Nov 05 03:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5984(8162984) WEXITSTATUS 1 Nov 05 03:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5984(8162984) done Nov 05 03:13:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6314(8163744) WEXITSTATUS 1 Nov 05 03:13:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6314(8163744) done Nov 05 03:13:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_520(8162406) WEXITSTATUS 0 Nov 05 03:13:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_520(8162406) done Nov 05 03:13:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_539(8162443) WEXITSTATUS 0 Nov 05 03:13:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_539(8162443) done Nov 05 03:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6367(8163995) WEXITSTATUS 1 Nov 05 03:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6367(8163995) done Nov 05 03:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_426(8162282) WEXITSTATUS 0 Nov 05 03:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_426(8162282) done Nov 05 03:13:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_11(8159077) WEXITSTATUS 0 Nov 05 03:13:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_11(8159077) done Nov 05 03:13:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1625(8154571) WEXITSTATUS 0 Nov 05 03:13:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1625(8154571) done Nov 05 03:13:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1331(8135432) WEXITSTATUS 0 Nov 05 03:13:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1331(8135432) done Nov 05 03:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1513(8143680) WEXITSTATUS 0 Nov 05 03:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1513(8143680) done Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:13:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:13:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159222 WEXITSTATUS 0 Nov 05 03:13:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159222 done Nov 05 03:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5638(8161840) WEXITSTATUS 1 Nov 05 03:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5638(8161840) done Nov 05 03:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_540(8162444) WEXITSTATUS 0 Nov 05 03:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_540(8162444) done Nov 05 03:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3730(8163992) WEXITSTATUS 0 Nov 05 03:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3730(8163992) done Nov 05 03:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_577(8162487) WEXITSTATUS 0 Nov 05 03:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_577(8162487) done Nov 05 03:13:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6366(8163957) WEXITSTATUS 1 Nov 05 03:13:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6366(8163957) done Nov 05 03:13:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1547(8146916) WEXITSTATUS 0 Nov 05 03:13:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1547(8146916) done Nov 05 03:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3736(8164129) WEXITSTATUS 0 Nov 05 03:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3736(8164129) done Nov 05 03:13:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_508(8162394) WEXITSTATUS 0 Nov 05 03:13:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_508(8162394) done Nov 05 03:14:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_466(8162336) WEXITSTATUS 0 Nov 05 03:14:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_466(8162336) done Nov 05 03:14:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3727(8163834) WEXITSTATUS 0 Nov 05 03:14:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3727(8163834) done Nov 05 03:14:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_579(8162489) WEXITSTATUS 0 Nov 05 03:14:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_579(8162489) done Nov 05 03:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3903(8145196) WEXITSTATUS 1 Nov 05 03:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3903(8145196) done Nov 05 03:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_609(8162533) WEXITSTATUS 0 Nov 05 03:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_609(8162533) done Nov 05 03:14:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5877(8162853) WEXITSTATUS 1 Nov 05 03:14:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5877(8162853) done Nov 05 03:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1378(8135679) WEXITSTATUS 0 Nov 05 03:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1378(8135679) done Nov 05 03:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1597(8152595) WEXITSTATUS 0 Nov 05 03:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1597(8152595) done Nov 05 03:14:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_516(8162402) WEXITSTATUS 0 Nov 05 03:14:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_516(8162402) done Nov 05 03:14:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:14:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_505(8162391) WEXITSTATUS 0 Nov 05 03:14:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_505(8162391) done Nov 05 03:14:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1487(8139217) WEXITSTATUS 0 Nov 05 03:14:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1487(8139217) done Nov 05 03:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4111(8149226) WEXITSTATUS 1 Nov 05 03:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4111(8149226) done Nov 05 03:14:28 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r815u35n06 reason set to: NHC: check_cmd_status: 142 (expected 0) returned by "test -e /vast/palmer/apps/.yalehpc". Nov 05 03:14:28 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r815u35n06 state set to DRAINING Nov 05 03:14:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 03:14:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_557(8162461) WEXITSTATUS 0 Nov 05 03:14:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_557(8162461) done Nov 05 03:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_517(8162403) WEXITSTATUS 0 Nov 05 03:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_517(8162403) done Nov 05 03:14:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_575(8162485) WEXITSTATUS 0 Nov 05 03:14:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_575(8162485) done Nov 05 03:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6323(8163753) WEXITSTATUS 1 Nov 05 03:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6323(8163753) done Nov 05 03:14:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_514(8162400) WEXITSTATUS 0 Nov 05 03:14:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_514(8162400) done Nov 05 03:14:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_595(8162505) WEXITSTATUS 0 Nov 05 03:14:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_595(8162505) done Nov 05 03:14:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154621 WEXITSTATUS 0 Nov 05 03:14:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154621 done Nov 05 03:14:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_534(8162438) WEXITSTATUS 0 Nov 05 03:14:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_534(8162438) done Nov 05 03:14:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_591(8162501) WEXITSTATUS 0 Nov 05 03:14:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_591(8162501) done Nov 05 03:14:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5923(8162918) WEXITSTATUS 1 Nov 05 03:14:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5923(8162918) done Nov 05 03:14:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_612(8162536) WEXITSTATUS 0 Nov 05 03:14:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_612(8162536) done Nov 05 03:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_512(8162398) WEXITSTATUS 0 Nov 05 03:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_512(8162398) done Nov 05 03:15:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5852(8162823) WEXITSTATUS 1 Nov 05 03:15:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5852(8162823) done Nov 05 03:15:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8163800 uid 25506 Nov 05 03:15:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_574(8162484) WEXITSTATUS 0 Nov 05 03:15:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_574(8162484) done Nov 05 03:15:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6139(8163180) WEXITSTATUS 1 Nov 05 03:15:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6139(8163180) done Nov 05 03:15:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5907(8162899) WEXITSTATUS 1 Nov 05 03:15:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5907(8162899) done Nov 05 03:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1374(8135669) WEXITSTATUS 0 Nov 05 03:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1374(8135669) done Nov 05 03:15:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133385 WEXITSTATUS 0 Nov 05 03:15:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133385 done Nov 05 03:15:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5778(8162728) WEXITSTATUS 1 Nov 05 03:15:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5778(8162728) done Nov 05 03:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_296(8162110) WEXITSTATUS 0 Nov 05 03:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_296(8162110) done Nov 05 03:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_503(8162389) WEXITSTATUS 0 Nov 05 03:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_503(8162389) done Nov 05 03:15:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_560(8162464) WEXITSTATUS 0 Nov 05 03:15:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_560(8162464) done Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:15:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:15:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1548(8146917) WEXITSTATUS 0 Nov 05 03:15:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1548(8146917) done Nov 05 03:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_611(8162535) WEXITSTATUS 0 Nov 05 03:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_611(8162535) done Nov 05 03:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_552(8162456) WEXITSTATUS 0 Nov 05 03:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_552(8162456) done Nov 05 03:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_460(8162330) WEXITSTATUS 0 Nov 05 03:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_460(8162330) done Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_547(8164143) in pi_ohern on r801u32n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_548(8164144) in pi_ohern on r917u13n01 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_549(8164145) in pi_ohern on r917u13n01 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_550(8164146) in pi_ohern on r917u13n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_551(8164147) in pi_ohern on r917u13n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_552(8164148) in pi_ohern on r917u13n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_553(8164149) in pi_ohern on r917u13n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_554(8164150) in pi_ohern on r917u13n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_555(8164151) in pi_ohern on r917u13n03 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_556(8164152) in pi_ohern on r917u13n03 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_557(8164153) in pi_ohern on r917u13n03 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_558(8164154) in pi_ohern on r917u29n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_559(8164155) in pi_ohern on r917u29n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_560(8164156) in pi_ohern on r917u29n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_561(8164157) in pi_ohern on r917u29n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_562(8164158) in pi_ohern on r917u29n02 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_563(8164159) in pi_ohern on r917u29n03 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_564(8164160) in pi_ohern on r917u29n03 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_565(8164161) in pi_ohern on r917u29n03 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_566(8164162) in pi_ohern on r917u29n04 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_567(8164163) in pi_ohern on r917u29n04 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_568(8164164) in pi_ohern on r917u29n04 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_569(8164165) in pi_ohern on r917u29n04 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_570(8164166) in pi_ohern on r917u29n04 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8128531 in day on r815u29n06 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3737(8164167) in gpu on r808u08n01 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3738(8164168) in gpu on r808u08n01 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3739(8164169) in gpu on r4513u29n01 Nov 05 03:15:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154708 in day on r919u01n01 Nov 05 03:15:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1994(8164170) in day on r914u01n02 Nov 05 03:15:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1995(8164171) in day on r918u01n03 Nov 05 03:15:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1996(8164172) in day on r919u09n03 Nov 05 03:15:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1997(8164173) in day on r816u09n05 Nov 05 03:15:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1998(8164174) in day on r816u09n08 Nov 05 03:15:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133398 in gpu on r908u24n02 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6402(8164175) in scavenge on r908u35n04 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6403(8164176) in scavenge on r903u04n03 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6404(8164177) in scavenge on r911u05n01 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6405(8164178) in scavenge on r802u02n03 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6406(8164179) in scavenge on r802u04n04 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6407(8164180) in scavenge on r802u11n01 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6408(8164181) in scavenge on r815u23n08 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6409(8164182) in scavenge on r816u15n10 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6410(8164183) in scavenge on r806u09n02 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6411(8164184) in scavenge on r806u09n02 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6412(8164185) in scavenge on r806u09n04 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6413(8164186) in scavenge on r909u04n04 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6414(8164187) in scavenge on r909u10n03 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6415(8164188) in scavenge on r912u36n02 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6416(8164189) in scavenge on r913u05n01 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_583(7572752) in scavenge on r815u29n06 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_175(8115645) in scavenge on r816u09n05 Nov 05 03:15:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_428(8114695) in scavenge on r816u23n04 Nov 05 03:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_553(8162457) WEXITSTATUS 0 Nov 05 03:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_553(8162457) done Nov 05 03:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_555(8162459) WEXITSTATUS 0 Nov 05 03:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_555(8162459) done Nov 05 03:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_515(8162401) WEXITSTATUS 0 Nov 05 03:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_515(8162401) done Nov 05 03:15:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_500(8162386) WEXITSTATUS 0 Nov 05 03:15:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_500(8162386) done Nov 05 03:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6301(8163697) WEXITSTATUS 1 Nov 05 03:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6301(8163697) done Nov 05 03:15:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133383 WEXITSTATUS 0 Nov 05 03:15:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133383 done Nov 05 03:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_518(8162404) WEXITSTATUS 0 Nov 05 03:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_518(8162404) done Nov 05 03:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1401(8135934) WEXITSTATUS 0 Nov 05 03:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1401(8135934) done Nov 05 03:16:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_554(8162458) WEXITSTATUS 0 Nov 05 03:16:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_554(8162458) done Nov 05 03:16:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1321(8135412) WEXITSTATUS 0 Nov 05 03:16:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1321(8135412) done Nov 05 03:16:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2305(8126261) Nov 05 03:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162641 WEXITSTATUS 0 Nov 05 03:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162641 done Nov 05 03:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6404(8164177) WEXITSTATUS 1 Nov 05 03:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6404(8164177) done Nov 05 03:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3737(8164167) WEXITSTATUS 0 Nov 05 03:16:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3737(8164167) done Nov 05 03:16:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6017(8163027) WEXITSTATUS 1 Nov 05 03:16:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6017(8163027) done Nov 05 03:16:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162690 NodeList=r815u03n08 #CPUs=16 Partition=day Nov 05 03:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6373(8164001) WEXITSTATUS 1 Nov 05 03:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6373(8164001) done Nov 05 03:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_499(8162385) WEXITSTATUS 0 Nov 05 03:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_499(8162385) done Nov 05 03:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1281(8135260) WEXITSTATUS 0 Nov 05 03:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1281(8135260) done Nov 05 03:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3738(8164168) WEXITSTATUS 0 Nov 05 03:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3738(8164168) done Nov 05 03:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4343(8154953) WEXITSTATUS 1 Nov 05 03:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4343(8154953) done Nov 05 03:16:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1359(8135622) WEXITSTATUS 0 Nov 05 03:16:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1359(8135622) done Nov 05 03:16:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_295(8162109) WEXITSTATUS 0 Nov 05 03:16:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_295(8162109) done Nov 05 03:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3734(8164087) WEXITSTATUS 0 Nov 05 03:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3734(8164087) done Nov 05 03:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5794(8162753) WEXITSTATUS 1 Nov 05 03:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5794(8162753) done Nov 05 03:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6413(8164186) WEXITSTATUS 1 Nov 05 03:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6413(8164186) done Nov 05 03:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5990(8162990) WEXITSTATUS 1 Nov 05 03:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5990(8162990) done Nov 05 03:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_584(8162494) WEXITSTATUS 0 Nov 05 03:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_584(8162494) done Nov 05 03:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_510(8162396) WEXITSTATUS 0 Nov 05 03:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_510(8162396) done Nov 05 03:17:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:17:04 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:17:05 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:17:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:17:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:17:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:17:08 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:17:09 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:17:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_513(8162399) WEXITSTATUS 0 Nov 05 03:17:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_513(8162399) done Nov 05 03:17:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159224 WEXITSTATUS 0 Nov 05 03:17:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159224 done Nov 05 03:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6211(8163363) WEXITSTATUS 1 Nov 05 03:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6211(8163363) done Nov 05 03:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1406(8135939) WEXITSTATUS 0 Nov 05 03:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1406(8135939) done Nov 05 03:17:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6038(8163060) WEXITSTATUS 1 Nov 05 03:17:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6038(8163060) done Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:17:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_14(8112369) WEXITSTATUS 1 Nov 05 03:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_14(8112369) done Nov 05 03:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_588(8162498) WEXITSTATUS 0 Nov 05 03:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_588(8162498) done Nov 05 03:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_602(8162512) WEXITSTATUS 0 Nov 05 03:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_602(8162512) done Nov 05 03:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1335(8135468) WEXITSTATUS 0 Nov 05 03:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1335(8135468) done Nov 05 03:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_598(8162508) WEXITSTATUS 0 Nov 05 03:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_598(8162508) done Nov 05 03:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1563(8149121) WEXITSTATUS 0 Nov 05 03:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1563(8149121) done Nov 05 03:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_530(8162434) WEXITSTATUS 0 Nov 05 03:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_530(8162434) done Nov 05 03:17:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_422(8162278) WEXITSTATUS 0 Nov 05 03:17:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_422(8162278) done Nov 05 03:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1592(8152162) WEXITSTATUS 0 Nov 05 03:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1592(8152162) done Nov 05 03:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_551(8162455) WEXITSTATUS 0 Nov 05 03:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_551(8162455) done Nov 05 03:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_616(8162540) WEXITSTATUS 0 Nov 05 03:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_616(8162540) done Nov 05 03:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6343(8163846) WEXITSTATUS 1 Nov 05 03:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6343(8163846) done Nov 05 03:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5876(8162852) WEXITSTATUS 1 Nov 05 03:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5876(8162852) done Nov 05 03:18:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159102 WEXITSTATUS 0 Nov 05 03:18:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159102 done Nov 05 03:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162679 WEXITSTATUS 0 Nov 05 03:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162679 done Nov 05 03:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1558(8148465) WEXITSTATUS 0 Nov 05 03:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1558(8148465) done Nov 05 03:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1599(8152802) WEXITSTATUS 0 Nov 05 03:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1599(8152802) done Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_571(8164190) in pi_ohern on r801u32n02 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_572(8164191) in pi_ohern on r917u13n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_573(8164192) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_574(8164193) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_575(8164194) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_576(8164195) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_577(8164196) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_578(8164197) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_579(8164198) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_580(8164199) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_581(8164200) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_582(8164201) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_583(8164202) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_584(8164203) in pi_ohern on r917u29n01 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_585(8164204) in pi_ohern on r917u29n02 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_586(8164205) in pi_ohern on r917u29n02 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_587(8164206) in pi_ohern on r917u29n03 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_588(8164207) in pi_ohern on r917u29n03 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_589(8164208) in pi_ohern on r917u29n03 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_590(8164209) in pi_ohern on r917u29n03 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_591(8164210) in pi_ohern on r917u29n04 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_592(8164211) in pi_ohern on r917u29n04 Nov 05 03:18:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_593(8164212) in pi_ohern on r917u29n04 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8162691 in day on r909u04n03 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3740(8164213) in gpu on r808u08n01 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3741(8164214) in gpu on r808u08n01 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3742(8164215) in gpu on r4513u23n01 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1999(8164216) in day on r806u04n01 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2000(8164217) in day on r909u01n03 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2001(8164218) in day on r914u05n03 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2002(8164219) in day on r918u01n02 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2003(8164220) in day on r919u05n03 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2004(8164221) in day on r919u05n04 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2005(8164222) in day on r815u29n05 Nov 05 03:18:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2006(8164223) in day on r816u15n01 Nov 05 03:18:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133399 in gpu on r908u24n02 Nov 05 03:18:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:18:21 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8157966 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6417(8164224) in scavenge on r907u01n04 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6418(8164225) in scavenge on r908u33n01 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6419(8164226) in scavenge on r802u11n03 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6420(8164227) in scavenge on r911u05n01 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6421(8164228) in scavenge on r917u01n02 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6422(8164229) in scavenge on r801u04n03 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6423(8164230) in scavenge on r801u11n04 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6424(8164231) in scavenge on r815u03n09 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6425(8164232) in scavenge on r909u04n04 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6426(8164233) in scavenge on r909u07n03 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6427(8164234) in scavenge on r918u01n03 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6428(8164235) in scavenge on r918u36n01 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6429(8164236) in scavenge on r919u05n04 Nov 05 03:18:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6430(8164237) in scavenge on r919u09n01 Nov 05 03:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_559(8162463) WEXITSTATUS 0 Nov 05 03:18:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_559(8162463) done Nov 05 03:18:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1542(8146145) WEXITSTATUS 0 Nov 05 03:18:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1542(8146145) done Nov 05 03:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_519(8162405) WEXITSTATUS 0 Nov 05 03:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_519(8162405) done Nov 05 03:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_583(8162493) WEXITSTATUS 0 Nov 05 03:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_583(8162493) done Nov 05 03:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6180(8163266) WEXITSTATUS 1 Nov 05 03:18:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6180(8163266) done Nov 05 03:18:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133384 WEXITSTATUS 0 Nov 05 03:18:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133384 done Nov 05 03:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6036(8163052) WEXITSTATUS 1 Nov 05 03:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6036(8163052) done Nov 05 03:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_573(8162483) WEXITSTATUS 0 Nov 05 03:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_573(8162483) done Nov 05 03:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_589(8162499) WEXITSTATUS 0 Nov 05 03:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_589(8162499) done Nov 05 03:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1486(8139216) WEXITSTATUS 0 Nov 05 03:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1486(8139216) done Nov 05 03:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3740(8164213) WEXITSTATUS 0 Nov 05 03:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3740(8164213) done Nov 05 03:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159221 WEXITSTATUS 0 Nov 05 03:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159221 done Nov 05 03:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_623(8162547) WEXITSTATUS 0 Nov 05 03:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_623(8162547) done Nov 05 03:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6401(8164142) WEXITSTATUS 1 Nov 05 03:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6401(8164142) done Nov 05 03:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_585(8162495) WEXITSTATUS 0 Nov 05 03:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_585(8162495) done Nov 05 03:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5817(8162779) WEXITSTATUS 1 Nov 05 03:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5817(8162779) done Nov 05 03:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_504(8162390) WEXITSTATUS 0 Nov 05 03:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_504(8162390) done Nov 05 03:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_601(8162511) WEXITSTATUS 0 Nov 05 03:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_601(8162511) done Nov 05 03:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6417(8164224) WEXITSTATUS 1 Nov 05 03:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6417(8164224) done Nov 05 03:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6064(8163087) WEXITSTATUS 1 Nov 05 03:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6064(8163087) done Nov 05 03:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6145(8163186) WEXITSTATUS 1 Nov 05 03:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6145(8163186) done Nov 05 03:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_13(8159159) WEXITSTATUS 0 Nov 05 03:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_13(8159159) done Nov 05 03:19:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:19:26 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r815u35n06 state set to ALLOCATED Nov 05 03:19:27 r912u23n01 slurmctld[163760]: slurmctld: Node r815u35n06 now responding Nov 05 03:19:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 03:19:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1587(8151847) WEXITSTATUS 0 Nov 05 03:19:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1587(8151847) done Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:19:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:19:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_423(8162279) WEXITSTATUS 0 Nov 05 03:19:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_423(8162279) done Nov 05 03:19:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_594(8162504) WEXITSTATUS 0 Nov 05 03:19:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_594(8162504) done Nov 05 03:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_592(8162502) WEXITSTATUS 0 Nov 05 03:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_592(8162502) done Nov 05 03:19:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_511(8162397) WEXITSTATUS 0 Nov 05 03:19:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_511(8162397) done Nov 05 03:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_610(8162534) WEXITSTATUS 0 Nov 05 03:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_610(8162534) done Nov 05 03:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_597(8162507) WEXITSTATUS 0 Nov 05 03:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_597(8162507) done Nov 05 03:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162659 WEXITSTATUS 0 Nov 05 03:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162659 done Nov 05 03:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7589443 WEXITSTATUS 0 Nov 05 03:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7589443 done Nov 05 03:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_586(8162496) WEXITSTATUS 0 Nov 05 03:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_586(8162496) done Nov 05 03:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6163(8163241) WEXITSTATUS 1 Nov 05 03:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6163(8163241) done Nov 05 03:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5891(8162867) WEXITSTATUS 1 Nov 05 03:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5891(8162867) done Nov 05 03:20:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6391(8164132) WEXITSTATUS 1 Nov 05 03:20:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6391(8164132) done Nov 05 03:20:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5128(8158468) WEXITSTATUS 1 Nov 05 03:20:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5128(8158468) done Nov 05 03:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_618(8162542) WEXITSTATUS 0 Nov 05 03:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_618(8162542) done Nov 05 03:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5816(8162778) WEXITSTATUS 1 Nov 05 03:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5816(8162778) done Nov 05 03:20:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_615(8162539) WEXITSTATUS 0 Nov 05 03:20:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_615(8162539) done Nov 05 03:20:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_633(8162570) WEXITSTATUS 0 Nov 05 03:20:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_633(8162570) done Nov 05 03:20:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3742(8164215) WEXITSTATUS 0 Nov 05 03:20:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3742(8164215) done Nov 05 03:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159922 WEXITSTATUS 0 Nov 05 03:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159922 done Nov 05 03:20:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 03:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6161(8163208) WEXITSTATUS 1 Nov 05 03:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6161(8163208) done Nov 05 03:20:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162692 NodeList=r805u06n01 #CPUs=16 Partition=day Nov 05 03:20:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8162693 NodeList=r918u32n01 #CPUs=16 Partition=day Nov 05 03:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6018(8163028) WEXITSTATUS 1 Nov 05 03:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6018(8163028) done Nov 05 03:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6095(8163128) WEXITSTATUS 1 Nov 05 03:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6095(8163128) done Nov 05 03:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_582(8162492) WEXITSTATUS 0 Nov 05 03:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_582(8162492) done Nov 05 03:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5795(8162754) WEXITSTATUS 1 Nov 05 03:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5795(8162754) done Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_594(8164238) in pi_ohern on r917u13n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_595(8164239) in pi_ohern on r917u13n02 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_596(8164240) in pi_ohern on r917u29n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_597(8164241) in pi_ohern on r917u29n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_598(8164242) in pi_ohern on r917u29n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_599(8164243) in pi_ohern on r917u29n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_600(8164244) in pi_ohern on r917u29n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_601(8164245) in pi_ohern on r917u29n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_602(8164246) in pi_ohern on r917u29n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_603(8164247) in pi_ohern on r917u29n02 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_604(8164248) in pi_ohern on r917u29n02 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_605(8164249) in pi_ohern on r917u29n02 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_606(8164250) in pi_ohern on r917u29n02 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_607(8164251) in pi_ohern on r917u29n02 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_608(8164252) in pi_ohern on r917u29n02 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_609(8164253) in pi_ohern on r917u29n03 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_610(8164254) in pi_ohern on r917u29n03 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_611(8164255) in pi_ohern on r917u29n03 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_612(8164256) in pi_ohern on r917u29n04 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_613(8164257) in pi_ohern on r917u29n04 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3743(8164258) in gpu on r808u08n01 Nov 05 03:20:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3744(8164259) in gpu on r4513u23n01 Nov 05 03:21:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2007(8164260) in day on r816u15n07 Nov 05 03:21:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133400 in gpu on r908u24n01 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6431(8164261) in scavenge on r907u01n04 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6432(8164262) in scavenge on r908u37n04 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6433(8164263) in scavenge on r908u39n02 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6434(8164264) in scavenge on r801u09n03 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6435(8164265) in scavenge on r802u06n03 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6436(8164266) in scavenge on r815u09n09 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6437(8164267) in scavenge on r816u15n11 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6438(8164268) in scavenge on r816u23n12 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6439(8164269) in scavenge on r909u29n03 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6440(8164270) in scavenge on r904u04n04 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6441(8164271) in scavenge on r806u04n03 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6442(8164272) in scavenge on r806u09n03 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6443(8164273) in scavenge on r806u09n03 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6444(8164274) in scavenge on r913u01n01 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6445(8164275) in scavenge on r913u05n04 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6446(8164276) in scavenge on r913u13n03 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_631(8114900) in scavenge on r816u09n03 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_733(8115002) in scavenge on r816u15n01 Nov 05 03:21:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_747(8115016) in scavenge on r816u15n07 Nov 05 03:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_631(8162555) WEXITSTATUS 0 Nov 05 03:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_631(8162555) done Nov 05 03:21:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1617(8154193) WEXITSTATUS 0 Nov 05 03:21:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1617(8154193) done Nov 05 03:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_649(8162586) WEXITSTATUS 0 Nov 05 03:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_649(8162586) done Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_617(8162541) WEXITSTATUS 0 Nov 05 03:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_617(8162541) done Nov 05 03:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163904 WEXITSTATUS 0 Nov 05 03:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163904 done Nov 05 03:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_619(8162543) WEXITSTATUS 0 Nov 05 03:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_619(8162543) done Nov 05 03:21:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_587(8162497) WEXITSTATUS 0 Nov 05 03:21:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_587(8162497) done Nov 05 03:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_596(8162506) WEXITSTATUS 0 Nov 05 03:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_596(8162506) done Nov 05 03:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3744(8164259) WEXITSTATUS 0 Nov 05 03:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3744(8164259) done Nov 05 03:21:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8164277 NodeList=(null) usec=25336 Nov 05 03:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8164277 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 03:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_599(8162509) WEXITSTATUS 0 Nov 05 03:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_599(8162509) done Nov 05 03:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1427(8135960) WEXITSTATUS 0 Nov 05 03:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1427(8135960) done Nov 05 03:22:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6414(8164187) WEXITSTATUS 1 Nov 05 03:22:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6414(8164187) done Nov 05 03:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5901(8162877) WEXITSTATUS 1 Nov 05 03:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5901(8162877) done Nov 05 03:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1620(8154566) WEXITSTATUS 0 Nov 05 03:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1620(8154566) done Nov 05 03:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_626(8162550) WEXITSTATUS 0 Nov 05 03:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_626(8162550) done Nov 05 03:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1388(8135779) WEXITSTATUS 0 Nov 05 03:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1388(8135779) done Nov 05 03:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_590(8162500) WEXITSTATUS 0 Nov 05 03:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_590(8162500) done Nov 05 03:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1526(8145082) WEXITSTATUS 0 Nov 05 03:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1526(8145082) done Nov 05 03:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1477(8138591) WEXITSTATUS 0 Nov 05 03:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1477(8138591) done Nov 05 03:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6408(8164181) WEXITSTATUS 1 Nov 05 03:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6408(8164181) done Nov 05 03:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_647(8162584) WEXITSTATUS 0 Nov 05 03:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_647(8162584) done Nov 05 03:22:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:22:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:22:53 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:22:54 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_630(8162554) WEXITSTATUS 0 Nov 05 03:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_630(8162554) done Nov 05 03:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1370(8135646) WEXITSTATUS 0 Nov 05 03:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1370(8135646) done Nov 05 03:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3715(8163396) WEXITSTATUS 0 Nov 05 03:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3715(8163396) done Nov 05 03:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5964(8162964) WEXITSTATUS 1 Nov 05 03:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5964(8162964) done Nov 05 03:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1555(8147566) WEXITSTATUS 0 Nov 05 03:23:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1555(8147566) done Nov 05 03:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6061(8163084) WEXITSTATUS 1 Nov 05 03:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6061(8163084) done Nov 05 03:23:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_593(8162503) WEXITSTATUS 0 Nov 05 03:23:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_593(8162503) done Nov 05 03:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_558(8162462) WEXITSTATUS 0 Nov 05 03:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_558(8162462) done Nov 05 03:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5949(8162945) WEXITSTATUS 1 Nov 05 03:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5949(8162945) done Nov 05 03:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6281(8163600) WEXITSTATUS 1 Nov 05 03:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6281(8163600) done Nov 05 03:23:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_608(8162532) WEXITSTATUS 0 Nov 05 03:23:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_608(8162532) done Nov 05 03:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_639(8162576) WEXITSTATUS 0 Nov 05 03:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_639(8162576) done Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:23:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_637(8162574) WEXITSTATUS 0 Nov 05 03:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_637(8162574) done Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_614(8164278) in pi_ohern on r917u13n01 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_615(8164279) in pi_ohern on r917u29n01 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_616(8164280) in pi_ohern on r917u29n01 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_617(8164281) in pi_ohern on r917u29n02 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_618(8164282) in pi_ohern on r917u29n02 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_619(8164283) in pi_ohern on r917u29n02 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_620(8164284) in pi_ohern on r917u29n02 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_621(8164285) in pi_ohern on r917u29n02 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_622(8164286) in pi_ohern on r917u29n02 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_623(8164287) in pi_ohern on r917u29n03 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_624(8164288) in pi_ohern on r917u29n03 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_625(8164289) in pi_ohern on r917u29n03 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_626(8164290) in pi_ohern on r917u29n04 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_627(8164291) in pi_ohern on r917u29n04 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_628(8164292) in pi_ohern on r917u29n04 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_629(8164293) in pi_ohern on r917u29n04 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3745(8164294) in gpu on r4513u23n01 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3746(8164295) in gpu on r4513u23n01 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2008(8164296) in day on r909u10n03 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2009(8164297) in day on r918u05n04 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2010(8164298) in day on r919u05n01 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2011(8164299) in day on r919u32n02 Nov 05 03:23:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2012(8164300) in day on r816u03n04 Nov 05 03:23:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_627(8164291) WEXITSTATUS 0 Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_627(8164291) done Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_628(8164292) WEXITSTATUS 0 Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_628(8164292) done Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_625(8164289) WEXITSTATUS 0 Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_625(8164289) done Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_626(8164290) WEXITSTATUS 0 Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_626(8164290) done Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_629(8164293) WEXITSTATUS 0 Nov 05 03:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_629(8164293) done Nov 05 03:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6447(8164301) in scavenge on r802u04n04 Nov 05 03:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6448(8164302) in scavenge on r815u23n08 Nov 05 03:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6449(8164303) in scavenge on r816u15n11 Nov 05 03:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6450(8164304) in scavenge on r816u35n12 Nov 05 03:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6451(8164305) in scavenge on r909u10n03 Nov 05 03:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6452(8164306) in scavenge on r918u05n04 Nov 05 03:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6453(8164307) in scavenge on r918u32n02 Nov 05 03:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_750(8115019) in scavenge on r908u24n02 Nov 05 03:23:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_622(8162546) WEXITSTATUS 0 Nov 05 03:23:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_622(8162546) done Nov 05 03:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_613(8162537) WEXITSTATUS 0 Nov 05 03:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_613(8162537) done Nov 05 03:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1584(8151559) WEXITSTATUS 0 Nov 05 03:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1584(8151559) done Nov 05 03:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6215(8163367) WEXITSTATUS 1 Nov 05 03:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6215(8163367) done Nov 05 03:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_648(8162585) WEXITSTATUS 0 Nov 05 03:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_648(8162585) done Nov 05 03:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_40(8135739) WEXITSTATUS 0 Nov 05 03:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_40(8135739) done Nov 05 03:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_640(8162577) WEXITSTATUS 0 Nov 05 03:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_640(8162577) done Nov 05 03:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_614(8162538) WEXITSTATUS 0 Nov 05 03:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_614(8162538) done Nov 05 03:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5912(8162904) WEXITSTATUS 1 Nov 05 03:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5912(8162904) done Nov 05 03:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1340(8135478) WEXITSTATUS 0 Nov 05 03:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1340(8135478) done Nov 05 03:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_642(8162579) WEXITSTATUS 0 Nov 05 03:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_642(8162579) done Nov 05 03:24:21 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2320(8126281) Nov 05 03:24:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_621(8162545) WEXITSTATUS 0 Nov 05 03:24:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_621(8162545) done Nov 05 03:24:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144130 NodeList=r802u27n03,r917u07n03 #CPUs=84 Partition=scavenge Nov 05 03:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1603(8153021) WEXITSTATUS 0 Nov 05 03:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1603(8153021) done Nov 05 03:24:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1426(8135959) WEXITSTATUS 0 Nov 05 03:24:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1426(8135959) done Nov 05 03:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4977(8158239) WEXITSTATUS 1 Nov 05 03:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4977(8158239) done Nov 05 03:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_600(8162510) WEXITSTATUS 0 Nov 05 03:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_600(8162510) done Nov 05 03:24:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6079(8163106) WEXITSTATUS 1 Nov 05 03:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6079(8163106) done Nov 05 03:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3706(8163301) WEXITSTATUS 0 Nov 05 03:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3706(8163301) done Nov 05 03:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_636(8162573) WEXITSTATUS 0 Nov 05 03:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_636(8162573) done Nov 05 03:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1614(8154047) WEXITSTATUS 0 Nov 05 03:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1614(8154047) done Nov 05 03:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6380(8164091) WEXITSTATUS 1 Nov 05 03:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6380(8164091) done Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:25:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:25:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_638(8162575) WEXITSTATUS 0 Nov 05 03:25:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_638(8162575) done Nov 05 03:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6201(8163337) WEXITSTATUS 1 Nov 05 03:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6201(8163337) done Nov 05 03:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3741(8164214) WEXITSTATUS 0 Nov 05 03:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3741(8164214) done Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_630(8164308) in pi_ohern on r917u29n01 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_631(8164309) in pi_ohern on r917u29n01 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_632(8164310) in pi_ohern on r917u29n02 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_633(8164311) in pi_ohern on r917u29n02 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_634(8164312) in pi_ohern on r917u29n02 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_635(8164313) in pi_ohern on r917u29n02 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_636(8164314) in pi_ohern on r917u29n02 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_637(8164315) in pi_ohern on r917u29n02 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_638(8164316) in pi_ohern on r917u29n03 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_639(8164317) in pi_ohern on r917u29n03 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_640(8164318) in pi_ohern on r917u29n04 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_641(8164319) in pi_ohern on r917u29n04 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_642(8164320) in pi_ohern on r917u29n04 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_643(8164321) in pi_ohern on r917u29n04 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_644(8164322) in pi_ohern on r917u29n04 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3747(8164323) in gpu on r808u08n01 Nov 05 03:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3748(8164324) in gpu on r808u08n01 Nov 05 03:26:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2013(8164325) in day on r918u09n03 Nov 05 03:26:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2014(8164326) in day on r815u15n04 Nov 05 03:26:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2015(8164327) in day on r815u23n06 Nov 05 03:26:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2016(8164328) in day on r815u35n05 Nov 05 03:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_636(8164314) WEXITSTATUS 0 Nov 05 03:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_636(8164314) done Nov 05 03:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_634(8164312) WEXITSTATUS 0 Nov 05 03:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_634(8164312) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_637(8164315) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_637(8164315) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_630(8164308) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_630(8164308) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_633(8164311) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_633(8164311) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_635(8164313) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_635(8164313) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_631(8164309) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_631(8164309) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_641(8164319) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_641(8164319) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_632(8164310) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_632(8164310) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_640(8164318) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_640(8164318) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_642(8164320) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_642(8164320) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_643(8164321) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_643(8164321) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_639(8164317) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_639(8164317) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_638(8164316) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_638(8164316) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_644(8164322) WEXITSTATUS 0 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_644(8164322) done Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6454(8164329) in scavenge on r903u01n01 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6455(8164330) in scavenge on r801u09n03 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6456(8164331) in scavenge on r801u09n04 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6457(8164332) in scavenge on r816u23n11 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6458(8164333) in scavenge on r816u35n07 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6459(8164334) in scavenge on r918u36n04 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6460(8164335) in scavenge on r919u09n01 Nov 05 03:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_810(8115079) in scavenge on r816u03n07 Nov 05 03:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1259(8135164) WEXITSTATUS 0 Nov 05 03:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1259(8135164) done Nov 05 03:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5868(8162840) WEXITSTATUS 1 Nov 05 03:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5868(8162840) done Nov 05 03:26:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1141(7572108) Nov 05 03:26:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_718(8114987) Nov 05 03:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_632(8162556) WEXITSTATUS 0 Nov 05 03:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_632(8162556) done Nov 05 03:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1421(8135954) WEXITSTATUS 0 Nov 05 03:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1421(8135954) done Nov 05 03:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_635(8162572) WEXITSTATUS 0 Nov 05 03:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_635(8162572) done Nov 05 03:27:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4902(8158035) WEXITSTATUS 1 Nov 05 03:27:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4902(8158035) done Nov 05 03:27:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6107(8163145) WEXITSTATUS 1 Nov 05 03:27:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6107(8163145) done Nov 05 03:27:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162636 WEXITSTATUS 0 Nov 05 03:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162636 done Nov 05 03:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3718(8163441) WEXITSTATUS 0 Nov 05 03:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3718(8163441) done Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:27:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164277 WEXITSTATUS 0 Nov 05 03:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164277 done Nov 05 03:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6438(8164268) WEXITSTATUS 1 Nov 05 03:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6438(8164268) done Nov 05 03:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_629(8162553) WEXITSTATUS 0 Nov 05 03:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_629(8162553) done Nov 05 03:27:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8164336 NodeList=(null) usec=22001 Nov 05 03:27:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8164336 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 03:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6393(8164134) WEXITSTATUS 1 Nov 05 03:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6393(8164134) done Nov 05 03:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6436(8164266) WEXITSTATUS 1 Nov 05 03:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6436(8164266) done Nov 05 03:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162660 WEXITSTATUS 0 Nov 05 03:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162660 done Nov 05 03:28:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_624(8162548) WEXITSTATUS 0 Nov 05 03:28:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_624(8162548) done Nov 05 03:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1616(8154192) WEXITSTATUS 0 Nov 05 03:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1616(8154192) done Nov 05 03:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6429(8164236) WEXITSTATUS 1 Nov 05 03:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6429(8164236) done Nov 05 03:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5751(8162616) WEXITSTATUS 1 Nov 05 03:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5751(8162616) done Nov 05 03:28:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:28:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 03:28:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_620(8162544) WEXITSTATUS 0 Nov 05 03:28:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_620(8162544) done Nov 05 03:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_627(8162551) WEXITSTATUS 0 Nov 05 03:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_627(8162551) done Nov 05 03:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1336(8135469) WEXITSTATUS 0 Nov 05 03:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1336(8135469) done Nov 05 03:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_628(8162552) WEXITSTATUS 0 Nov 05 03:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_628(8162552) done Nov 05 03:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1357(8135620) WEXITSTATUS 0 Nov 05 03:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1357(8135620) done Nov 05 03:28:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5328(8158901) WEXITSTATUS 1 Nov 05 03:28:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5328(8158901) done Nov 05 03:28:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5526(8160705) WEXITSTATUS 1 Nov 05 03:28:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5526(8160705) done Nov 05 03:28:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5961(8162961) WEXITSTATUS 1 Nov 05 03:28:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5961(8162961) done Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_645(8164337) in pi_ohern on r917u29n01 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_646(8164338) in pi_ohern on r917u29n01 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_647(8164339) in pi_ohern on r917u29n01 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_648(8164340) in pi_ohern on r917u29n02 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_649(8164341) in pi_ohern on r917u29n02 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_650(8164342) in pi_ohern on r917u29n02 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_651(8164343) in pi_ohern on r917u29n02 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_652(8164344) in pi_ohern on r917u29n02 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_653(8164345) in pi_ohern on r917u29n02 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_654(8164346) in pi_ohern on r917u29n02 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_655(8164347) in pi_ohern on r917u29n03 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_656(8164348) in pi_ohern on r917u29n03 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_657(8164349) in pi_ohern on r917u29n03 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_658(8164350) in pi_ohern on r917u29n03 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_659(8164351) in pi_ohern on r917u29n03 Nov 05 03:29:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_660(8164352) in pi_ohern on r917u29n03 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_661(8164353) in pi_ohern on r917u29n04 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_662(8164354) in pi_ohern on r917u29n04 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_663(8164355) in pi_ohern on r917u29n04 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_664(8164356) in pi_ohern on r917u29n04 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_665(8164357) in pi_ohern on r917u29n04 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_666(8164358) in pi_ohern on r917u29n04 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3749(8164359) in gpu on r903u34n01 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2017(8164360) in day on r805u14n04 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2018(8164361) in day on r909u13n01 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2019(8164362) in day on r919u01n04 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2020(8164363) in day on r919u05n03 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2021(8164364) in day on r815u09n03 Nov 05 03:29:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2022(8164365) in day on r816u03n07 Nov 05 03:29:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_665(8164357) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_665(8164357) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_651(8164343) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_651(8164343) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_645(8164337) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_645(8164337) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_666(8164358) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_666(8164358) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_647(8164339) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_647(8164339) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_663(8164355) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_663(8164355) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154632 WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154632 done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_660(8164352) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_660(8164352) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_658(8164350) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_658(8164350) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_652(8164344) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_652(8164344) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_662(8164354) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_662(8164354) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_650(8164342) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_650(8164342) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_653(8164345) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_653(8164345) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_654(8164346) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_654(8164346) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_655(8164347) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_655(8164347) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_661(8164353) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_661(8164353) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_649(8164341) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_649(8164341) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_646(8164338) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_646(8164338) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_664(8164356) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_664(8164356) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_648(8164340) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_648(8164340) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_656(8164348) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_656(8164348) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_659(8164351) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_659(8164351) done Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_657(8164349) WEXITSTATUS 0 Nov 05 03:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_657(8164349) done Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6461(8164366) in scavenge on r908u35n02 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6462(8164367) in scavenge on r801u06n01 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6463(8164368) in scavenge on r815u09n09 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6464(8164369) in scavenge on r816u23n12 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6465(8164370) in scavenge on r816u35n10 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6466(8164371) in scavenge on r808u08n01 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6467(8164372) in scavenge on r912u32n03 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6468(8164373) in scavenge on r805u04n01 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6469(8164374) in scavenge on r805u14n04 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6470(8164375) in scavenge on r805u25n02 Nov 05 03:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6471(8164376) in scavenge on r918u05n02 Nov 05 03:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5766(8162702) WEXITSTATUS 1 Nov 05 03:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5766(8162702) done Nov 05 03:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159936 WEXITSTATUS 0 Nov 05 03:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159936 done Nov 05 03:29:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_641(8162578) WEXITSTATUS 0 Nov 05 03:29:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_641(8162578) done Nov 05 03:29:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162640 WEXITSTATUS 0 Nov 05 03:29:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162640 done Nov 05 03:29:21 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158045 Nov 05 03:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_634(8162571) WEXITSTATUS 0 Nov 05 03:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_634(8162571) done Nov 05 03:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1522(8144002) WEXITSTATUS 0 Nov 05 03:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1522(8144002) done Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:29:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6439(8164269) WEXITSTATUS 1 Nov 05 03:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6439(8164269) done Nov 05 03:30:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6422(8164229) WEXITSTATUS 1 Nov 05 03:30:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6422(8164229) done Nov 05 03:30:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_625(8162549) WEXITSTATUS 0 Nov 05 03:30:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_625(8162549) done Nov 05 03:30:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5508(8160511) WEXITSTATUS 1 Nov 05 03:30:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5508(8160511) done Nov 05 03:30:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6072(8163099) WEXITSTATUS 1 Nov 05 03:30:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6072(8163099) done Nov 05 03:30:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:30:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 03:30:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n04 #CPUs=1 Partition=day Nov 05 03:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6333(8163797) WEXITSTATUS 1 Nov 05 03:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6333(8163797) done Nov 05 03:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6196(8163318) WEXITSTATUS 1 Nov 05 03:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6196(8163318) done Nov 05 03:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1442(8135975) WEXITSTATUS 0 Nov 05 03:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1442(8135975) done Nov 05 03:30:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 03:30:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 03:30:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 03:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6140(8163181) WEXITSTATUS 1 Nov 05 03:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6140(8163181) done Nov 05 03:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5808(8162768) WEXITSTATUS 1 Nov 05 03:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5808(8162768) done Nov 05 03:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2102(8125913) WEXITSTATUS 1 Nov 05 03:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2102(8125913) done Nov 05 03:31:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6394(8164135) WEXITSTATUS 1 Nov 05 03:31:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6394(8164135) done Nov 05 03:31:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6111(8163149) WEXITSTATUS 1 Nov 05 03:31:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6111(8163149) done Nov 05 03:31:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_643(8162580) WEXITSTATUS 0 Nov 05 03:31:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_643(8162580) done Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:31:32 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:31:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_667(8164377) in pi_ohern on r917u29n01 Nov 05 03:31:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159119_668(8159119) in pi_ohern on r917u29n01 Nov 05 03:31:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154712 in day on r816u09n05 Nov 05 03:31:46 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 03:31:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2023(8164378) in day on r909u07n03 Nov 05 03:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_668(8159119) WEXITSTATUS 0 Nov 05 03:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_668(8159119) done Nov 05 03:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_667(8164377) WEXITSTATUS 0 Nov 05 03:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_667(8164377) done Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6472(8164379) in scavenge on r801u04n03 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6473(8164380) in scavenge on r802u02n03 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6474(8164381) in scavenge on r802u25n01 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6475(8164382) in scavenge on r816u15n09 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6476(8164383) in scavenge on r4513u29n01 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6477(8164384) in scavenge on r903u34n01 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6478(8164385) in scavenge on r805u25n02 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6479(8164386) in scavenge on r913u24n02 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6480(8164387) in scavenge on r914u01n01 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6481(8164388) in scavenge on r909u01n03 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6482(8164389) in scavenge on r909u07n03 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6483(8164390) in scavenge on r909u13n02 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_850(8115119) in scavenge on r917u29n02 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1168(7572135) in scavenge on r917u29n03 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_244(8114510) in scavenge on r815u15n02 Nov 05 03:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_365(8114632) in scavenge on r815u15n02 Nov 05 03:31:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5767(8162703) WEXITSTATUS 1 Nov 05 03:31:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5767(8162703) done Nov 05 03:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159933 WEXITSTATUS 0 Nov 05 03:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159933 done Nov 05 03:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1346(8135491) WEXITSTATUS 0 Nov 05 03:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1346(8135491) done Nov 05 03:32:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1632(8155339) WEXITSTATUS 0 Nov 05 03:32:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1632(8155339) done Nov 05 03:32:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6425(8164232) WEXITSTATUS 1 Nov 05 03:32:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6425(8164232) done Nov 05 03:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1611(8153862) WEXITSTATUS 0 Nov 05 03:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1611(8153862) done Nov 05 03:32:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_45(8135744) WEXITSTATUS 0 Nov 05 03:32:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_45(8135744) done Nov 05 03:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1163(8133200) WEXITSTATUS 0 Nov 05 03:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1163(8133200) done Nov 05 03:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5758(8162628) WEXITSTATUS 1 Nov 05 03:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5758(8162628) done Nov 05 03:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1591(8152161) WEXITSTATUS 0 Nov 05 03:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1591(8152161) done Nov 05 03:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6447(8164301) WEXITSTATUS 1 Nov 05 03:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6447(8164301) done Nov 05 03:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5471(8159894) WEXITSTATUS 1 Nov 05 03:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5471(8159894) done Nov 05 03:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_644(8162581) WEXITSTATUS 0 Nov 05 03:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_644(8162581) done Nov 05 03:33:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3747(8164323) WEXITSTATUS 0 Nov 05 03:33:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3747(8164323) done Nov 05 03:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1534(8146012) WEXITSTATUS 0 Nov 05 03:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1534(8146012) done Nov 05 03:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5020(8158311) WEXITSTATUS 1 Nov 05 03:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5020(8158311) done Nov 05 03:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6054(8163077) WEXITSTATUS 1 Nov 05 03:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6054(8163077) done Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:33:33 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1583(8151558) WEXITSTATUS 0 Nov 05 03:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1583(8151558) done Nov 05 03:33:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_645(8162582) WEXITSTATUS 0 Nov 05 03:33:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_645(8162582) done Nov 05 03:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3745(8164294) WEXITSTATUS 0 Nov 05 03:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3745(8164294) done Nov 05 03:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6214(8163366) WEXITSTATUS 1 Nov 05 03:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6214(8163366) done Nov 05 03:34:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1622(8154568) WEXITSTATUS 0 Nov 05 03:34:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1622(8154568) done Nov 05 03:34:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1315(8135392) WEXITSTATUS 0 Nov 05 03:34:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1315(8135392) done Nov 05 03:34:21 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158091 Nov 05 03:34:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3750(8164391) in gpu on r808u08n01 Nov 05 03:34:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3751(8164392) in gpu on r4513u23n01 Nov 05 03:34:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2024(8164393) in day on r912u36n01 Nov 05 03:34:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2025(8164394) in day on r919u01n02 Nov 05 03:34:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2026(8164395) in day on r816u09n06 Nov 05 03:34:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2027(8164396) in day on r816u15n01 Nov 05 03:34:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2028(8164397) in day on r815u23n01 Nov 05 03:34:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2029(8164398) in day on r815u29n05 Nov 05 03:34:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2030(8164399) in day on r815u35n02 Nov 05 03:34:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n04 Nov 05 03:34:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6484(8164400) in scavenge on r907u05n02 Nov 05 03:34:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6485(8164401) in scavenge on r908u35n01 Nov 05 03:34:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6486(8164402) in scavenge on r802u04n04 Nov 05 03:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6487(8164403) in scavenge on r802u09n03 Nov 05 03:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6488(8164404) in scavenge on r806u14n01 Nov 05 03:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6489(8164405) in scavenge on r807u03n04 Nov 05 03:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6490(8164406) in scavenge on r912u36n01 Nov 05 03:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6491(8164407) in scavenge on r912u36n01 Nov 05 03:34:34 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2267896960]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 05 03:34:34 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2267896960]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 05 03:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_646(8162583) WEXITSTATUS 0 Nov 05 03:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159029_646(8162583) done Nov 05 03:34:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2341(8126314) Nov 05 03:34:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1265(8135192) WEXITSTATUS 0 Nov 05 03:34:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1265(8135192) done Nov 05 03:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6269(8163551) WEXITSTATUS 1 Nov 05 03:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6269(8163551) done Nov 05 03:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1562(8149120) WEXITSTATUS 0 Nov 05 03:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1562(8149120) done Nov 05 03:35:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6230(8163411) WEXITSTATUS 1 Nov 05 03:35:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6230(8163411) done Nov 05 03:35:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3735(8164128) WEXITSTATUS 0 Nov 05 03:35:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3735(8164128) done Nov 05 03:35:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3720(8163483) WEXITSTATUS 0 Nov 05 03:35:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3720(8163483) done Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:35:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4621(8156205) WEXITSTATUS 1 Nov 05 03:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4621(8156205) done Nov 05 03:35:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162637 WEXITSTATUS 0 Nov 05 03:35:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162637 done Nov 05 03:35:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:36:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3750(8164391) WEXITSTATUS 0 Nov 05 03:36:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3750(8164391) done Nov 05 03:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1566(8149340) WEXITSTATUS 0 Nov 05 03:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1566(8149340) done Nov 05 03:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6183(8163282) WEXITSTATUS 1 Nov 05 03:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6183(8163282) done Nov 05 03:36:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5908(8162900) WEXITSTATUS 1 Nov 05 03:36:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5908(8162900) done Nov 05 03:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2533(8152691) WEXITSTATUS 0 Nov 05 03:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2533(8152691) done Nov 05 03:37:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162680 WEXITSTATUS 0 Nov 05 03:37:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162680 done Nov 05 03:37:09 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3752(8164408) in gpu on r808u08n01 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3753(8164409) in gpu on r808u08n01 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3754(8164410) in gpu on r903u40n01 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2031(8164411) in day on r913u28n04 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2032(8164412) in day on r918u05n02 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2033(8164413) in day on r918u09n03 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2034(8164414) in day on r815u23n05 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2035(8164415) in day on r816u03n03 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6229(8163410) WEXITSTATUS 1 Nov 05 03:37:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6229(8163410) done Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6492(8164416) in scavenge on r908u33n02 Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6493(8164417) in scavenge on r801u14n01 Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6494(8164418) in scavenge on r904u01n03 Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6495(8164419) in scavenge on r816u35n11 Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6496(8164420) in scavenge on r816u09n10 Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6497(8164421) in scavenge on r816u15n10 Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6498(8164422) in scavenge on r912u36n01 Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6499(8164423) in scavenge on r909u04n04 Nov 05 03:37:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_817(8115086) in scavenge on r816u03n03 Nov 05 03:37:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907588 WEXITSTATUS 0 Nov 05 03:37:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907588 done Nov 05 03:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162645 WEXITSTATUS 0 Nov 05 03:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162645 done Nov 05 03:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1377(8135678) WEXITSTATUS 0 Nov 05 03:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1377(8135678) done Nov 05 03:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3754(8164410) WEXITSTATUS 0 Nov 05 03:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3754(8164410) done Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6464(8164369) WEXITSTATUS 1 Nov 05 03:37:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6464(8164369) done Nov 05 03:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5700(8162362) WEXITSTATUS 1 Nov 05 03:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5700(8162362) done Nov 05 03:37:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6420(8164227) WEXITSTATUS 1 Nov 05 03:37:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6420(8164227) done Nov 05 03:37:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6316(8163746) WEXITSTATUS 1 Nov 05 03:37:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6316(8163746) done Nov 05 03:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6462(8164367) WEXITSTATUS 1 Nov 05 03:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6462(8164367) done Nov 05 03:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 03:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 03:38:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 03:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1398(8135840) WEXITSTATUS 0 Nov 05 03:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1398(8135840) done Nov 05 03:38:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1471(8138472) WEXITSTATUS 0 Nov 05 03:38:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1471(8138472) done Nov 05 03:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3751(8164392) WEXITSTATUS 0 Nov 05 03:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3751(8164392) done Nov 05 03:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6488(8164404) WEXITSTATUS 1 Nov 05 03:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6488(8164404) done Nov 05 03:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1535(8146013) WEXITSTATUS 0 Nov 05 03:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1535(8146013) done Nov 05 03:39:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5654(8161947) WEXITSTATUS 1 Nov 05 03:39:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5654(8161947) done Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6430(8164237) WEXITSTATUS 1 Nov 05 03:39:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6430(8164237) done Nov 05 03:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6475(8164382) WEXITSTATUS 1 Nov 05 03:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6475(8164382) done Nov 05 03:39:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907589 in gpu on r805u34n01 Nov 05 03:39:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3755(8164424) in gpu on r4513u23n01 Nov 05 03:39:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3756(8164425) in gpu on r903u40n01 Nov 05 03:39:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2036(8164426) in day on r906u41n04 Nov 05 03:39:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2037(8164427) in day on r909u10n01 Nov 05 03:39:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2038(8164428) in day on r909u13n03 Nov 05 03:39:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:39:54 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2345(8126318) Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6500(8164429) in scavenge on r801u06n01 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6501(8164430) in scavenge on r802u11n01 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6502(8164431) in scavenge on r911u05n01 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6503(8164432) in scavenge on r912u05n04 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6504(8164433) in scavenge on r816u15n09 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6505(8164434) in scavenge on r816u23n12 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6506(8164435) in scavenge on r806u14n01 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6507(8164436) in scavenge on r906u41n04 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6508(8164437) in scavenge on r906u41n04 Nov 05 03:39:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_860(7573029) in scavenge on r816u29n02 Nov 05 03:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5803(8162762) WEXITSTATUS 1 Nov 05 03:39:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5803(8162762) done Nov 05 03:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1607(8153646) WEXITSTATUS 0 Nov 05 03:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1607(8153646) done Nov 05 03:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5666(8162100) WEXITSTATUS 1 Nov 05 03:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5666(8162100) done Nov 05 03:40:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3755(8164424) WEXITSTATUS 0 Nov 05 03:40:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3755(8164424) done Nov 05 03:40:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:40:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6396(8164137) WEXITSTATUS 1 Nov 05 03:40:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6396(8164137) done Nov 05 03:41:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3746(8164295) WEXITSTATUS 0 Nov 05 03:41:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3746(8164295) done Nov 05 03:41:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8133368 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:41:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:41:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6305(8163701) WEXITSTATUS 1 Nov 05 03:41:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6305(8163701) done Nov 05 03:42:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5464(8159887) WEXITSTATUS 1 Nov 05 03:42:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5464(8159887) done Nov 05 03:42:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3757(8164438) in gpu on r4513u23n01 Nov 05 03:42:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3758(8164439) in gpu on r4513u23n01 Nov 05 03:42:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2039(8164440) in day on r805u25n03 Nov 05 03:42:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2040(8164441) in day on r815u29n05 Nov 05 03:42:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6509(8164442) in scavenge on r802u23n02 Nov 05 03:42:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6510(8164443) in scavenge on r904u01n03 Nov 05 03:42:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6511(8164444) in scavenge on r816u03n10 Nov 05 03:42:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6512(8164445) in scavenge on r805u25n03 Nov 05 03:42:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6513(8164446) in scavenge on r805u25n03 Nov 05 03:42:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1112(7573281) in scavenge on r805u25n03 Nov 05 03:42:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5603(8161690) WEXITSTATUS 1 Nov 05 03:42:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5603(8161690) done Nov 05 03:42:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5818(8162780) WEXITSTATUS 1 Nov 05 03:42:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5818(8162780) done Nov 05 03:42:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_927(7573096) Nov 05 03:42:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_309(8114575) Nov 05 03:42:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2569(8152836) WEXITSTATUS 0 Nov 05 03:42:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2569(8152836) done Nov 05 03:43:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6093(8163126) WEXITSTATUS 1 Nov 05 03:43:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6093(8163126) done Nov 05 03:43:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163213 WEXITSTATUS 0 Nov 05 03:43:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163213 done Nov 05 03:43:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1609(8153860) WEXITSTATUS 0 Nov 05 03:43:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1609(8153860) done Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:43:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3758(8164439) WEXITSTATUS 0 Nov 05 03:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3758(8164439) done Nov 05 03:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6295(8163648) WEXITSTATUS 1 Nov 05 03:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6295(8163648) done Nov 05 03:44:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163212 WEXITSTATUS 0 Nov 05 03:44:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163212 done Nov 05 03:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1638(8155796) WEXITSTATUS 0 Nov 05 03:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1638(8155796) done Nov 05 03:44:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163210 WEXITSTATUS 0 Nov 05 03:44:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163210 done Nov 05 03:44:26 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158181 Nov 05 03:44:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1635(8155685) WEXITSTATUS 0 Nov 05 03:44:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1635(8155685) done Nov 05 03:44:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133374 WEXITSTATUS 0 Nov 05 03:44:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133374 done Nov 05 03:44:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6128(8163169) WEXITSTATUS 1 Nov 05 03:44:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6128(8163169) done Nov 05 03:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5022(8158313) WEXITSTATUS 1 Nov 05 03:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5022(8158313) done Nov 05 03:45:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3759(8164447) in gpu on r4513u23n01 Nov 05 03:45:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2041(8164448) in day on r815u15n08 Nov 05 03:45:11 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 03:45:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133401 in gpu on r806u41n01 Nov 05 03:45:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:45:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6514(8164449) in scavenge on r802u11n02 Nov 05 03:45:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6515(8164450) in scavenge on r815u23n07 Nov 05 03:45:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6516(8164451) in scavenge on r907u28n01 Nov 05 03:45:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6517(8164452) in scavenge on r909u13n04 Nov 05 03:45:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6518(8164453) in scavenge on r909u29n03 Nov 05 03:45:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6519(8164454) in scavenge on r913u01n02 Nov 05 03:45:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_795(8115064) in scavenge on r913u28n01 Nov 05 03:45:17 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 03:45:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1627(8154573) WEXITSTATUS 0 Nov 05 03:45:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1627(8154573) done Nov 05 03:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5235(8158638) WEXITSTATUS 1 Nov 05 03:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5235(8158638) done Nov 05 03:45:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_883(8115152) Nov 05 03:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6495(8164419) WEXITSTATUS 1 Nov 05 03:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6495(8164419) done Nov 05 03:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1581(8151039) WEXITSTATUS 0 Nov 05 03:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1581(8151039) done Nov 05 03:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1618(8154372) WEXITSTATUS 0 Nov 05 03:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1618(8154372) done Nov 05 03:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2689(8153076) WEXITSTATUS 0 Nov 05 03:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2689(8153076) done Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:45:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5309(8158771) WEXITSTATUS 1 Nov 05 03:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5309(8158771) done Nov 05 03:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3759(8164447) WEXITSTATUS 0 Nov 05 03:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3759(8164447) done Nov 05 03:46:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154646 WEXITSTATUS 0 Nov 05 03:46:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154646 done Nov 05 03:46:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_692(8126246) WEXITSTATUS 0 Nov 05 03:46:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_692(8126246) done Nov 05 03:46:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6445(8164275) WEXITSTATUS 1 Nov 05 03:46:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6445(8164275) done Nov 05 03:46:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163209 WEXITSTATUS 0 Nov 05 03:46:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163209 done Nov 05 03:46:28 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:46:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6483(8164390) WEXITSTATUS 1 Nov 05 03:46:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6483(8164390) done Nov 05 03:47:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133372 WEXITSTATUS 0 Nov 05 03:47:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133372 done Nov 05 03:47:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162644 WEXITSTATUS 0 Nov 05 03:47:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162644 done Nov 05 03:47:27 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158216 Nov 05 03:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6504(8164433) WEXITSTATUS 1 Nov 05 03:47:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6504(8164433) done Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:47:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4913(8158076) WEXITSTATUS 1 Nov 05 03:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4913(8158076) done Nov 05 03:47:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3760(8164455) in gpu on r4513u23n01 Nov 05 03:47:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154709 in day on r918u36n04 Nov 05 03:47:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2042(8164456) in day on r815u15n03 Nov 05 03:47:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2043(8164457) in day on r815u35n04 Nov 05 03:47:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133509 in gpu on r806u41n01 Nov 05 03:47:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 03:47:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1040(8128644) WEXITSTATUS 0 Nov 05 03:47:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1040(8128644) done Nov 05 03:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6520(8164458) in scavenge on r802u06n02 Nov 05 03:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6521(8164459) in scavenge on r816u15n09 Nov 05 03:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6522(8164460) in scavenge on r816u35n11 Nov 05 03:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6523(8164461) in scavenge on r909u13n02 Nov 05 03:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6524(8164462) in scavenge on r909u13n04 Nov 05 03:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6525(8164463) in scavenge on r912u32n03 Nov 05 03:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6526(8164464) in scavenge on r913u05n04 Nov 05 03:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_914(7573083) in scavenge on r815u35n04 Nov 05 03:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133509 WEXITSTATUS 1 Nov 05 03:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133509 done Nov 05 03:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6353(8163897) WEXITSTATUS 1 Nov 05 03:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6353(8163897) done Nov 05 03:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6216(8163368) WEXITSTATUS 1 Nov 05 03:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6216(8163368) done Nov 05 03:48:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159940 WEXITSTATUS 0 Nov 05 03:48:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159940 done Nov 05 03:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6248(8163489) WEXITSTATUS 1 Nov 05 03:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6248(8163489) done Nov 05 03:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162690 WEXITSTATUS 0 Nov 05 03:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162690 done Nov 05 03:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1582(8151243) WEXITSTATUS 0 Nov 05 03:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1582(8151243) done Nov 05 03:48:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6262(8163544) WEXITSTATUS 1 Nov 05 03:48:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6262(8163544) done Nov 05 03:49:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3733(8164086) WEXITSTATUS 0 Nov 05 03:49:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3733(8164086) done Nov 05 03:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5411(8159392) WEXITSTATUS 1 Nov 05 03:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5411(8159392) done Nov 05 03:49:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159934 WEXITSTATUS 0 Nov 05 03:49:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159934 done Nov 05 03:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1594(8152389) WEXITSTATUS 0 Nov 05 03:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1594(8152389) done Nov 05 03:49:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159937 WEXITSTATUS 0 Nov 05 03:49:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159937 done Nov 05 03:49:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159939 WEXITSTATUS 0 Nov 05 03:49:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159939 done Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:49:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:49:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6157(8163204) WEXITSTATUS 1 Nov 05 03:49:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6157(8163204) done Nov 05 03:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5701(8162363) WEXITSTATUS 1 Nov 05 03:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5701(8162363) done Nov 05 03:50:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6506(8164435) WEXITSTATUS 1 Nov 05 03:50:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6506(8164435) done Nov 05 03:50:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5841(8162808) WEXITSTATUS 1 Nov 05 03:50:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5841(8162808) done Nov 05 03:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3760(8164455) WEXITSTATUS 0 Nov 05 03:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3760(8164455) done Nov 05 03:50:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5983(8162983) WEXITSTATUS 1 Nov 05 03:50:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5983(8162983) done Nov 05 03:50:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164465 InitPrio=25712 usec=4974 Nov 05 03:50:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1553(8147297) WEXITSTATUS 0 Nov 05 03:50:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1553(8147297) done Nov 05 03:50:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8164465 in day on r904u07n02 Nov 05 03:50:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3761(8164466) in gpu on r808u14n01 Nov 05 03:50:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3762(8164467) in gpu on r4513u23n01 Nov 05 03:50:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2044(8164468) in day on r815u29n04 Nov 05 03:50:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2045(8164469) in day on r816u09n08 Nov 05 03:50:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2046(8164470) in day on r815u15n03 Nov 05 03:50:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2047(8164471) in day on r815u15n07 Nov 05 03:50:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133510 in gpu on r806u41n01 Nov 05 03:50:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6527(8164472) in scavenge on r801u06n01 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6528(8164473) in scavenge on r802u02n02 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6529(8164474) in scavenge on r903u07n01 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6530(8164475) in scavenge on r906u39n01 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6531(8164476) in scavenge on r806u14n01 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6532(8164477) in scavenge on r909u07n04 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6533(8164478) in scavenge on r914u01n02 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6534(8164479) in scavenge on r913u09n03 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6535(8164480) in scavenge on r913u24n04 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6536(8164481) in scavenge on r918u01n03 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_12(8114278) in scavenge on r815u15n03 Nov 05 03:50:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1068(8115337) in scavenge on r816u35n02 Nov 05 03:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133510 WEXITSTATUS 1 Nov 05 03:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133510 done Nov 05 03:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6490(8164406) WEXITSTATUS 1 Nov 05 03:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6490(8164406) done Nov 05 03:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6529(8164474) WEXITSTATUS 1 Nov 05 03:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6529(8164474) done Nov 05 03:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6533(8164478) WEXITSTATUS 1 Nov 05 03:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6533(8164478) done Nov 05 03:50:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2434(8152478) WEXITSTATUS 0 Nov 05 03:50:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2434(8152478) done Nov 05 03:50:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6092(8163125) WEXITSTATUS 1 Nov 05 03:50:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6092(8163125) done Nov 05 03:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6144(8163185) WEXITSTATUS 1 Nov 05 03:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6144(8163185) done Nov 05 03:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1595(8152593) WEXITSTATUS 0 Nov 05 03:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1595(8152593) done Nov 05 03:51:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1142(8132313) WEXITSTATUS 0 Nov 05 03:51:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1142(8132313) done Nov 05 03:51:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651307_0(7651367) Nov 05 03:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162684 WEXITSTATUS 0 Nov 05 03:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162684 done Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:51:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1564(8149122) WEXITSTATUS 0 Nov 05 03:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1564(8149122) done Nov 05 03:51:58 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:52:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1363(8135635) WEXITSTATUS 0 Nov 05 03:52:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1363(8135635) done Nov 05 03:52:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6491(8164407) WEXITSTATUS 1 Nov 05 03:52:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6491(8164407) done Nov 05 03:52:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1605(8153241) WEXITSTATUS 0 Nov 05 03:52:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1605(8153241) done Nov 05 03:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5290(8158738) WEXITSTATUS 1 Nov 05 03:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5290(8158738) done Nov 05 03:53:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2048(8164482) in day on r806u04n03 Nov 05 03:53:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2049(8164483) in day on r909u13n02 Nov 05 03:53:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2050(8164484) in day on r816u15n03 Nov 05 03:53:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2051(8164485) in day on r815u29n03 Nov 05 03:53:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2052(8164486) in day on r815u35n04 Nov 05 03:53:13 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 03:53:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133511 in gpu on r806u41n01 Nov 05 03:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6537(8164487) in scavenge on r903u07n01 Nov 05 03:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6538(8164488) in scavenge on r807u03n04 Nov 05 03:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6539(8164489) in scavenge on r909u13n04 Nov 05 03:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6540(8164490) in scavenge on r912u36n01 Nov 05 03:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6541(8164491) in scavenge on r912u36n01 Nov 05 03:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6542(8164492) in scavenge on r914u01n02 Nov 05 03:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6543(8164493) in scavenge on r918u05n03 Nov 05 03:53:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_841(7573010) in scavenge on r816u09n03 Nov 05 03:53:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133511 WEXITSTATUS 1 Nov 05 03:53:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133511 done Nov 05 03:53:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5969(8162969) WEXITSTATUS 1 Nov 05 03:53:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5969(8162969) done Nov 05 03:53:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7651307_0(7651367) NodeList=r815u35n07 #CPUs=1 Partition=scavenge_gpu Nov 05 03:53:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3519(8158224) WEXITSTATUS 0 Nov 05 03:53:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3519(8158224) done Nov 05 03:53:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_29(8161546) WEXITSTATUS 0 Nov 05 03:53:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_29(8161546) done Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:53:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:53:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6238(8163444) WEXITSTATUS 1 Nov 05 03:53:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6238(8163444) done Nov 05 03:54:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6350(8163853) WEXITSTATUS 1 Nov 05 03:54:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6350(8163853) done Nov 05 03:54:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5814(8162774) WEXITSTATUS 1 Nov 05 03:54:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5814(8162774) done Nov 05 03:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5730(8162563) WEXITSTATUS 1 Nov 05 03:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5730(8162563) done Nov 05 03:54:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5833(8162800) WEXITSTATUS 1 Nov 05 03:54:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5833(8162800) done Nov 05 03:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_33(8161682) WEXITSTATUS 0 Nov 05 03:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_33(8161682) done Nov 05 03:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6208(8163344) WEXITSTATUS 1 Nov 05 03:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6208(8163344) done Nov 05 03:55:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6179(8163265) WEXITSTATUS 1 Nov 05 03:55:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6179(8163265) done Nov 05 03:55:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3761(8164466) WEXITSTATUS 0 Nov 05 03:55:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3761(8164466) done Nov 05 03:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6456(8164331) WEXITSTATUS 1 Nov 05 03:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6456(8164331) done Nov 05 03:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5884(8162860) WEXITSTATUS 1 Nov 05 03:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5884(8162860) done Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:55:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3749(8164359) WEXITSTATUS 0 Nov 05 03:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3749(8164359) done Nov 05 03:55:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3763(8164494) in gpu on r808u14n01 Nov 05 03:55:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3764(8164495) in gpu on r808u14n01 Nov 05 03:55:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3765(8164496) in gpu on r903u34n01 Nov 05 03:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133375 WEXITSTATUS 0 Nov 05 03:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133375 done Nov 05 03:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6045(8163067) WEXITSTATUS 1 Nov 05 03:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6045(8163067) done Nov 05 03:55:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133512 in gpu on r806u41n01 Nov 05 03:55:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133513 in gpu on r915u09n01 Nov 05 03:55:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 03:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133512 WEXITSTATUS 1 Nov 05 03:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133512 done Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6544(8164497) in scavenge on r801u09n04 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6545(8164498) in scavenge on r911u24n03 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6546(8164499) in scavenge on r915u32n01 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6547(8164500) in scavenge on r815u23n08 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6548(8164501) in scavenge on r815u23n09 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6549(8164502) in scavenge on r815u35n07 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6550(8164503) in scavenge on r805u27n02 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6551(8164504) in scavenge on r805u27n02 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6552(8164505) in scavenge on r807u05n01 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6553(8164506) in scavenge on r807u05n01 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6554(8164507) in scavenge on r904u07n01 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_884(7573053) in scavenge on r904u04n01 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1188(7573357) in scavenge on r805u34n01 Nov 05 03:55:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_842(7573011) in scavenge on r805u34n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2053(8164508) in scavenge on r917u10n04 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2054(8164509) in scavenge on r801u23n03 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2055(8164510) in scavenge on r801u27n02 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2056(8164511) in scavenge on r911u05n02 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2057(8164512) in scavenge on r912u05n04 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2058(8164513) in scavenge on r815u23n07 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2059(8164514) in scavenge on r815u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2060(8164515) in scavenge on r815u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2061(8164516) in scavenge on r815u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2062(8164517) in scavenge on r815u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2063(8164518) in scavenge on r815u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2064(8164519) in scavenge on r815u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2065(8164520) in scavenge on r815u09n11 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2066(8164521) in scavenge on r815u09n11 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2067(8164522) in scavenge on r815u09n11 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2068(8164523) in scavenge on r815u15n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2069(8164524) in scavenge on r815u15n10 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2070(8164525) in scavenge on r815u15n10 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2071(8164526) in scavenge on r815u15n10 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2072(8164527) in scavenge on r816u29n07 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2073(8164528) in scavenge on r816u29n07 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2074(8164529) in scavenge on r816u29n08 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2075(8164530) in scavenge on r816u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2076(8164531) in scavenge on r816u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2077(8164532) in scavenge on r816u35n09 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2078(8164533) in scavenge on r816u35n10 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2079(8164534) in scavenge on r816u35n10 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2080(8164535) in scavenge on r816u35n10 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2081(8164536) in scavenge on r816u35n10 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2082(8164537) in scavenge on r816u35n12 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2083(8164538) in scavenge on r816u35n12 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2084(8164539) in scavenge on r816u35n12 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2085(8164540) in scavenge on r912u09n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2086(8164541) in scavenge on r912u09n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2087(8164542) in scavenge on r912u09n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2088(8164543) in scavenge on r912u13n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2089(8164544) in scavenge on r912u13n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2090(8164545) in scavenge on r912u13n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2091(8164546) in scavenge on r912u13n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2092(8164547) in scavenge on r808u08n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2093(8164548) in scavenge on r907u34n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2094(8164549) in scavenge on r816u35n07 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2095(8164550) in scavenge on r816u35n07 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2096(8164551) in scavenge on r906u28n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2097(8164552) in scavenge on r816u23n03 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2098(8164553) in scavenge on r816u29n03 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2099(8164554) in scavenge on r816u29n06 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2100(8164555) in scavenge on r816u29n06 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2101(8164556) in scavenge on r816u35n04 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2102(8164557) in scavenge on r816u35n05 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2103(8164558) in scavenge on r816u35n05 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2104(8164559) in scavenge on r816u35n05 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2105(8164560) in scavenge on r806u31n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2106(8164561) in scavenge on r806u31n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2107(8164562) in scavenge on r806u31n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2108(8164563) in scavenge on r906u13n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2109(8164564) in scavenge on r906u13n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2110(8164565) in scavenge on r906u32n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2111(8164566) in scavenge on r906u32n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2112(8164567) in scavenge on r906u32n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2113(8164568) in scavenge on r805u36n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2114(8164569) in scavenge on r805u36n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2115(8164570) in scavenge on r805u38n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2116(8164571) in scavenge on r805u38n01 Nov 05 03:56:02 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:128 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 03:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6541(8164491) WEXITSTATUS 1 Nov 05 03:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6541(8164491) done Nov 05 03:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6501(8164430) WEXITSTATUS 1 Nov 05 03:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6501(8164430) done Nov 05 03:56:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133513 WEXITSTATUS 1 Nov 05 03:56:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133513 done Nov 05 03:56:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162665 WEXITSTATUS 0 Nov 05 03:56:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162665 done Nov 05 03:56:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5994(8162999) WEXITSTATUS 1 Nov 05 03:56:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5994(8162999) done Nov 05 03:56:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6349(8163852) WEXITSTATUS 1 Nov 05 03:56:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6349(8163852) done Nov 05 03:56:59 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 03:57:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5887(8162863) WEXITSTATUS 1 Nov 05 03:57:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5887(8162863) done Nov 05 03:57:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162686 WEXITSTATUS 0 Nov 05 03:57:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162686 done Nov 05 03:57:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6423(8164230) WEXITSTATUS 1 Nov 05 03:57:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6423(8164230) done Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:57:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1330(8135431) WEXITSTATUS 0 Nov 05 03:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1330(8135431) done Nov 05 03:57:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6517(8164452) WEXITSTATUS 1 Nov 05 03:57:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6517(8164452) done Nov 05 03:57:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3764(8164495) WEXITSTATUS 0 Nov 05 03:57:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3764(8164495) done Nov 05 03:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1642(8155910) WEXITSTATUS 0 Nov 05 03:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1642(8155910) done Nov 05 03:58:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1399(8135841) WEXITSTATUS 0 Nov 05 03:58:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1399(8135841) done Nov 05 03:58:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162682 WEXITSTATUS 0 Nov 05 03:58:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162682 done Nov 05 03:58:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3766(8164572) in gpu on r808u14n01 Nov 05 03:58:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2117(8164573) in day on r909u07n01 Nov 05 03:58:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2118(8164574) in day on r909u07n04 Nov 05 03:58:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2119(8164575) in day on r918u09n02 Nov 05 03:58:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2120(8164576) in day on r919u01n04 Nov 05 03:58:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 03:58:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133514 in gpu on r4513u26n01 Nov 05 03:58:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133515 in gpu on r806u41n01 Nov 05 03:58:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162688 WEXITSTATUS 0 Nov 05 03:58:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162688 done Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6555(8164577) in scavenge on r907u05n04 Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6556(8164578) in scavenge on r801u11n04 Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6557(8164579) in scavenge on r802u11n01 Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6558(8164580) in scavenge on r815u29n11 Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6559(8164581) in scavenge on r909u07n01 Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6560(8164582) in scavenge on r909u13n04 Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6561(8164583) in scavenge on r912u36n01 Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_546(8114813) in scavenge on r918u36n01 Nov 05 03:58:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1021(7573190) in scavenge on r816u23n02 Nov 05 03:58:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133514 WEXITSTATUS 1 Nov 05 03:58:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133514 done Nov 05 03:58:57 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2364(8126379) Nov 05 03:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133515 WEXITSTATUS 1 Nov 05 03:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133515 done Nov 05 03:59:28 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158300 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 03:59:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 03:59:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6379(8164050) WEXITSTATUS 1 Nov 05 03:59:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6379(8164050) done Nov 05 03:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6346(8163849) WEXITSTATUS 1 Nov 05 03:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6346(8163849) done Nov 05 03:59:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1314(8135386) WEXITSTATUS 0 Nov 05 03:59:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1314(8135386) done Nov 05 04:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 04:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 04:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 04:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 04:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 04:00:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1626(8154572) WEXITSTATUS 0 Nov 05 04:00:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1626(8154572) done Nov 05 04:00:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6351(8163895) WEXITSTATUS 1 Nov 05 04:00:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6351(8163895) done Nov 05 04:00:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6319(8163749) WEXITSTATUS 1 Nov 05 04:00:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6319(8163749) done Nov 05 04:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_28(8161545) WEXITSTATUS 0 Nov 05 04:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_28(8161545) done Nov 05 04:00:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 04:00:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 04:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3766(8164572) WEXITSTATUS 0 Nov 05 04:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3766(8164572) done Nov 05 04:00:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5965(8162965) WEXITSTATUS 1 Nov 05 04:00:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5965(8162965) done Nov 05 04:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 04:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 04:00:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 04:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_993(8128379) WEXITSTATUS 0 Nov 05 04:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_993(8128379) done Nov 05 04:01:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3767(8164584) in gpu on r808u14n01 Nov 05 04:01:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2121(8164585) in day on r807u08n03 Nov 05 04:01:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2122(8164586) in day on r815u15n02 Nov 05 04:01:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2123(8164587) in day on r816u03n03 Nov 05 04:01:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 04:01:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133516 in gpu on r4513u26n01 Nov 05 04:01:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133517 in gpu on r806u41n01 Nov 05 04:01:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:01:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6562(8164588) in scavenge on r802u30n03 Nov 05 04:01:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6563(8164589) in scavenge on r915u35n04 Nov 05 04:01:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6564(8164590) in scavenge on r816u29n10 Nov 05 04:01:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6565(8164591) in scavenge on r805u25n04 Nov 05 04:01:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6566(8164592) in scavenge on r805u25n04 Nov 05 04:01:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6567(8164593) in scavenge on r806u04n04 Nov 05 04:01:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1128(7573297) in scavenge on r807u08n03 Nov 05 04:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133516 WEXITSTATUS 1 Nov 05 04:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133516 done Nov 05 04:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3767(8164584) WEXITSTATUS 0 Nov 05 04:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3767(8164584) done Nov 05 04:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5978(8162978) WEXITSTATUS 1 Nov 05 04:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5978(8162978) done Nov 05 04:01:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2632(8152899) WEXITSTATUS 0 Nov 05 04:01:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2632(8152899) done Nov 05 04:01:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164594 InitPrio=17389 usec=6862 Nov 05 04:01:32 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8164594 NodeList=r801u04n02 #CPUs=6 Partition=pi_esi Nov 05 04:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162689 WEXITSTATUS 0 Nov 05 04:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162689 done Nov 05 04:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164594 WEXITSTATUS 0 Nov 05 04:01:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164594 done Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:01:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:01:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6234(8163415) WEXITSTATUS 1 Nov 05 04:01:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6234(8163415) done Nov 05 04:02:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133517 WEXITSTATUS 1 Nov 05 04:02:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133517 done Nov 05 04:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159197 WEXITSTATUS 0 Nov 05 04:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159197 done Nov 05 04:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1628(8154935) WEXITSTATUS 0 Nov 05 04:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1628(8154935) done Nov 05 04:02:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6377(8164048) WEXITSTATUS 1 Nov 05 04:02:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6377(8164048) done Nov 05 04:02:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3742(8141900) WEXITSTATUS 1 Nov 05 04:02:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3742(8141900) done Nov 05 04:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6530(8164475) WEXITSTATUS 1 Nov 05 04:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6530(8164475) done Nov 05 04:02:29 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3763(8164494) WEXITSTATUS 0 Nov 05 04:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3763(8164494) done Nov 05 04:02:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162664 WEXITSTATUS 0 Nov 05 04:02:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162664 done Nov 05 04:02:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6449(8164303) WEXITSTATUS 1 Nov 05 04:02:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6449(8164303) done Nov 05 04:02:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6141(8163182) WEXITSTATUS 1 Nov 05 04:02:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6141(8163182) done Nov 05 04:02:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_37(8161709) WEXITSTATUS 0 Nov 05 04:02:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_37(8161709) done Nov 05 04:02:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164595 InitPrio=17389 usec=5046 Nov 05 04:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6532(8164477) WEXITSTATUS 1 Nov 05 04:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6532(8164477) done Nov 05 04:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162681 WEXITSTATUS 0 Nov 05 04:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162681 done Nov 05 04:03:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6152(8163199) WEXITSTATUS 1 Nov 05 04:03:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6152(8163199) done Nov 05 04:03:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6563(8164589) WEXITSTATUS 1 Nov 05 04:03:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6563(8164589) done Nov 05 04:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1631(8155208) WEXITSTATUS 0 Nov 05 04:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1631(8155208) done Nov 05 04:03:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159200 WEXITSTATUS 0 Nov 05 04:03:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159200 done Nov 05 04:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2543(8152810) WEXITSTATUS 0 Nov 05 04:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2543(8152810) done Nov 05 04:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_32(8161681) WEXITSTATUS 0 Nov 05 04:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_32(8161681) done Nov 05 04:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 04:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 04:03:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 04:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1643(8155917) WEXITSTATUS 0 Nov 05 04:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1643(8155917) done Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:03:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162685 WEXITSTATUS 0 Nov 05 04:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162685 done Nov 05 04:03:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164596 InitPrio=17389 usec=4452 Nov 05 04:03:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8164596 NodeList=r801u04n02 #CPUs=12 Partition=pi_esi Nov 05 04:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6209(8163345) WEXITSTATUS 1 Nov 05 04:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6209(8163345) done Nov 05 04:03:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3768(8164597) in gpu on r808u14n01 Nov 05 04:03:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3769(8164598) in gpu on r808u14n01 Nov 05 04:03:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2124(8164599) in day on r909u04n04 Nov 05 04:03:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2125(8164600) in day on r815u35n06 Nov 05 04:03:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2126(8164601) in day on r816u03n02 Nov 05 04:03:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2127(8164602) in day on r816u03n08 Nov 05 04:03:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2128(8164603) in day on r816u15n07 Nov 05 04:03:57 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2379(8126400) Nov 05 04:03:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8133518 in gpu on r4513u26n01 Nov 05 04:03:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164604 InitPrio=17389 usec=4522 Nov 05 04:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3762(8164467) WEXITSTATUS 0 Nov 05 04:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3762(8164467) done Nov 05 04:04:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133518 WEXITSTATUS 1 Nov 05 04:04:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133518 done Nov 05 04:04:00 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8164604 NodeList=r801u04n04 #CPUs=12 Partition=pi_esi Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6568(8164605) in scavenge on r915u35n04 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6569(8164606) in scavenge on r816u15n11 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6570(8164607) in scavenge on r816u29n11 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6571(8164608) in scavenge on r906u39n01 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6572(8164609) in scavenge on r806u06n03 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6573(8164610) in scavenge on r806u06n03 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6574(8164611) in scavenge on r913u01n03 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6575(8164612) in scavenge on r904u10n02 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6576(8164613) in scavenge on r904u10n02 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6577(8164614) in scavenge on r904u10n02 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6578(8164615) in scavenge on r805u06n04 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6579(8164616) in scavenge on r805u11n03 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_369(8114636) in scavenge on r815u23n04 Nov 05 04:04:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_409(8114676) in scavenge on r816u03n08 Nov 05 04:04:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5811(8162771) WEXITSTATUS 1 Nov 05 04:04:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5811(8162771) done Nov 05 04:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164596 WEXITSTATUS 0 Nov 05 04:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164596 done Nov 05 04:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6197(8163319) WEXITSTATUS 1 Nov 05 04:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6197(8163319) done Nov 05 04:04:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164604 WEXITSTATUS 0 Nov 05 04:04:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164604 done Nov 05 04:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907438 WEXITSTATUS 0 Nov 05 04:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907438 done Nov 05 04:04:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_1438(8135971) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:04:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4725(8156736) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:04:28 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4754(8157104) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:04:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4725(8156736) Nov 05 04:04:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_1438(8135971) Nov 05 04:04:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4754(8157104) Nov 05 04:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6509(8164442) WEXITSTATUS 1 Nov 05 04:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6509(8164442) done Nov 05 04:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6579(8164616) WEXITSTATUS 1 Nov 05 04:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6579(8164616) done Nov 05 04:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6336(8163839) WEXITSTATUS 1 Nov 05 04:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6336(8163839) done Nov 05 04:04:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6313(8163709) WEXITSTATUS 1 Nov 05 04:04:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6313(8163709) done Nov 05 04:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_36(8161708) WEXITSTATUS 0 Nov 05 04:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_36(8161708) done Nov 05 04:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5665(8162099) WEXITSTATUS 1 Nov 05 04:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5665(8162099) done Nov 05 04:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1641(8155878) WEXITSTATUS 0 Nov 05 04:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1641(8155878) done Nov 05 04:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1651(8156175) WEXITSTATUS 0 Nov 05 04:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1651(8156175) done Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:05:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:05:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_31(8161680) WEXITSTATUS 0 Nov 05 04:05:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_31(8161680) done Nov 05 04:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1637(8155795) WEXITSTATUS 0 Nov 05 04:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1637(8155795) done Nov 05 04:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6257(8163498) WEXITSTATUS 1 Nov 05 04:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6257(8163498) done Nov 05 04:06:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6168(8163254) WEXITSTATUS 1 Nov 05 04:06:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6168(8163254) done Nov 05 04:06:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_34(8161706) WEXITSTATUS 0 Nov 05 04:06:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_34(8161706) done Nov 05 04:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1079(8130756) WEXITSTATUS 0 Nov 05 04:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1079(8130756) done Nov 05 04:06:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8164595 in gpu on r805u36n01 Nov 05 04:06:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907439 in gpu on r805u38n01 Nov 05 04:06:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3770(8164617) in gpu on r4513u23n01 Nov 05 04:06:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2129(8164618) in day on r815u03n06 Nov 05 04:06:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2130(8164619) in day on r815u23n02 Nov 05 04:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6580(8164620) in scavenge on r907u01n04 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6581(8164621) in scavenge on r908u35n03 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6582(8164622) in scavenge on r802u06n04 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6583(8164623) in scavenge on r802u23n02 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6584(8164624) in scavenge on r802u30n04 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6585(8164625) in scavenge on r806u09n02 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6586(8164626) in scavenge on r807u11n04 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6587(8164627) in scavenge on r913u24n03 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6588(8164628) in scavenge on r913u24n03 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6589(8164629) in scavenge on r805u06n04 Nov 05 04:06:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6590(8164630) in scavenge on r805u06n04 Nov 05 04:06:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_471(8114738) in scavenge on r805u09n03 Nov 05 04:06:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_219(8115689) in scavenge on r805u11n03 Nov 05 04:06:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_260(7571225) in scavenge on r805u27n03 Nov 05 04:06:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_1438(8135971) in scavenge on r815u15n10 Nov 05 04:06:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5639(8161841) WEXITSTATUS 1 Nov 05 04:06:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5639(8161841) done Nov 05 04:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6193(8163315) WEXITSTATUS 1 Nov 05 04:07:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6193(8163315) done Nov 05 04:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3757(8164438) WEXITSTATUS 0 Nov 05 04:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3757(8164438) done Nov 05 04:07:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6200(8163336) WEXITSTATUS 1 Nov 05 04:07:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6200(8163336) done Nov 05 04:07:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4655(8156317) WEXITSTATUS 1 Nov 05 04:07:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4655(8156317) done Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:07:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:07:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3770(8164617) WEXITSTATUS 0 Nov 05 04:07:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3770(8164617) done Nov 05 04:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6552(8164505) WEXITSTATUS 1 Nov 05 04:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6552(8164505) done Nov 05 04:08:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_31(8135730) WEXITSTATUS 0 Nov 05 04:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_31(8135730) done Nov 05 04:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6119(8163160) WEXITSTATUS 1 Nov 05 04:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6119(8163160) done Nov 05 04:08:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6398(8164139) WEXITSTATUS 1 Nov 05 04:08:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6398(8164139) done Nov 05 04:08:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6297(8163650) WEXITSTATUS 1 Nov 05 04:08:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6297(8163650) done Nov 05 04:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159199 WEXITSTATUS 0 Nov 05 04:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159199 done Nov 05 04:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6457(8164332) WEXITSTATUS 1 Nov 05 04:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6457(8164332) done Nov 05 04:09:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6384(8164095) WEXITSTATUS 1 Nov 05 04:09:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6384(8164095) done Nov 05 04:09:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3771(8164631) in gpu on r4513u23n01 Nov 05 04:09:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3772(8164632) in gpu on r4513u23n01 Nov 05 04:09:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6324(8163754) WEXITSTATUS 1 Nov 05 04:09:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6324(8163754) done Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6591(8164633) in scavenge on r916u01n01 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6592(8164634) in scavenge on r801u04n01 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6593(8164635) in scavenge on r801u23n01 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6594(8164636) in scavenge on r816u23n11 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6595(8164637) in scavenge on r816u29n07 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6596(8164638) in scavenge on r904u04n04 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6597(8164639) in scavenge on r807u05n01 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6598(8164640) in scavenge on r807u11n01 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6599(8164641) in scavenge on r805u04n02 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6600(8164642) in scavenge on r805u23n04 Nov 05 04:09:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6601(8164643) in scavenge on r805u27n03 Nov 05 04:09:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6109(8163147) WEXITSTATUS 1 Nov 05 04:09:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6109(8163147) done Nov 05 04:09:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2131(8164644) in scavenge on r816u23n01 Nov 05 04:09:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2132(8164645) in scavenge on r816u29n02 Nov 05 04:09:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2133(8164646) in scavenge on r816u29n02 Nov 05 04:09:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2134(8164647) in scavenge on r816u35n01 Nov 05 04:09:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2135(8164648) in scavenge on r816u35n03 Nov 05 04:09:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2136(8164649) in scavenge on r906u32n01 Nov 05 04:09:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3772(8164632) WEXITSTATUS 0 Nov 05 04:09:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3772(8164632) done Nov 05 04:09:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162678 WEXITSTATUS 0 Nov 05 04:09:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162678 done Nov 05 04:09:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6556(8164578) WEXITSTATUS 1 Nov 05 04:09:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6556(8164578) done Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:09:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:10:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5462(8159778) WEXITSTATUS 1 Nov 05 04:10:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5462(8159778) done Nov 05 04:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_39(8161743) WEXITSTATUS 0 Nov 05 04:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_39(8161743) done Nov 05 04:10:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162650 WEXITSTATUS 0 Nov 05 04:10:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162650 done Nov 05 04:10:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6186(8163285) WEXITSTATUS 1 Nov 05 04:10:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6186(8163285) done Nov 05 04:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6159(8163206) WEXITSTATUS 1 Nov 05 04:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6159(8163206) done Nov 05 04:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3768(8164597) WEXITSTATUS 0 Nov 05 04:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3768(8164597) done Nov 05 04:10:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163211 WEXITSTATUS 0 Nov 05 04:10:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8163211 done Nov 05 04:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1640(8155849) WEXITSTATUS 0 Nov 05 04:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1640(8155849) done Nov 05 04:11:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6385(8164096) WEXITSTATUS 1 Nov 05 04:11:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6385(8164096) done Nov 05 04:11:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159932 WEXITSTATUS 0 Nov 05 04:11:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159932 done Nov 05 04:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162643 WEXITSTATUS 0 Nov 05 04:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162643 done Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6587(8164627) WEXITSTATUS 1 Nov 05 04:11:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6587(8164627) done Nov 05 04:11:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3773(8164650) in gpu on r808u14n01 Nov 05 04:11:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3774(8164651) in gpu on r4513u23n01 Nov 05 04:11:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2137(8164652) in day on r909u29n02 Nov 05 04:11:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2138(8164653) in day on r815u23n03 Nov 05 04:11:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2139(8164654) in day on r816u03n06 Nov 05 04:11:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2140(8164655) in day on r816u09n02 Nov 05 04:11:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2141(8164656) in day on r816u09n03 Nov 05 04:11:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2385(8126410) Nov 05 04:11:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158395 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6602(8164657) in scavenge on r908u33n03 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6603(8164658) in scavenge on r801u11n04 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6604(8164659) in scavenge on r805u02n01 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6605(8164660) in scavenge on r805u02n01 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6606(8164661) in scavenge on r904u10n01 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6607(8164662) in scavenge on r806u02n03 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6608(8164663) in scavenge on r806u04n04 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6609(8164664) in scavenge on r912u28n04 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_318(7571283) in scavenge on r913u24n03 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_416(7571381) in scavenge on r816u03n06 Nov 05 04:12:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_559(7571524) in scavenge on r816u09n02 Nov 05 04:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162687 WEXITSTATUS 0 Nov 05 04:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162687 done Nov 05 04:12:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5935(8162930) WEXITSTATUS 1 Nov 05 04:12:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5935(8162930) done Nov 05 04:12:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6479(8164386) WEXITSTATUS 1 Nov 05 04:12:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6479(8164386) done Nov 05 04:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6099(8163132) WEXITSTATUS 1 Nov 05 04:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6099(8163132) done Nov 05 04:13:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_42(8161933) WEXITSTATUS 0 Nov 05 04:13:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_42(8161933) done Nov 05 04:13:01 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:13:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6291(8163644) WEXITSTATUS 1 Nov 05 04:13:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6291(8163644) done Nov 05 04:13:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6605(8164660) WEXITSTATUS 1 Nov 05 04:13:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6605(8164660) done Nov 05 04:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6283(8163602) WEXITSTATUS 1 Nov 05 04:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6283(8163602) done Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:13:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3773(8164650) WEXITSTATUS 0 Nov 05 04:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3773(8164650) done Nov 05 04:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_40(8161744) WEXITSTATUS 0 Nov 05 04:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_40(8161744) done Nov 05 04:13:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6113(8163151) WEXITSTATUS 1 Nov 05 04:13:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6113(8163151) done Nov 05 04:13:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5762(8162632) WEXITSTATUS 1 Nov 05 04:13:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5762(8162632) done Nov 05 04:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5221(8158608) WEXITSTATUS 1 Nov 05 04:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5221(8158608) done Nov 05 04:14:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1624(8154570) WEXITSTATUS 0 Nov 05 04:14:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1624(8154570) done Nov 05 04:14:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3774(8164651) WEXITSTATUS 0 Nov 05 04:14:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3774(8164651) done Nov 05 04:14:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5966(8162966) WEXITSTATUS 1 Nov 05 04:14:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5966(8162966) done Nov 05 04:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162651 WEXITSTATUS 0 Nov 05 04:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162651 done Nov 05 04:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6173(8163259) WEXITSTATUS 1 Nov 05 04:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6173(8163259) done Nov 05 04:14:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1601(8153019) WEXITSTATUS 0 Nov 05 04:14:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1601(8153019) done Nov 05 04:14:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3769(8164598) WEXITSTATUS 0 Nov 05 04:14:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3769(8164598) done Nov 05 04:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6604(8164659) WEXITSTATUS 1 Nov 05 04:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6604(8164659) done Nov 05 04:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2489(8152647) WEXITSTATUS 0 Nov 05 04:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2489(8152647) done Nov 05 04:14:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3775(8164665) in gpu on r808u14n01 Nov 05 04:14:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3776(8164666) in gpu on r808u14n01 Nov 05 04:14:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3777(8164667) in gpu on r4513u23n01 Nov 05 04:14:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2142(8164668) in day on r912u36n03 Nov 05 04:14:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2143(8164669) in day on r815u29n02 Nov 05 04:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5584(8161404) WEXITSTATUS 1 Nov 05 04:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5584(8161404) done Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6610(8164670) in scavenge on r908u28n02 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6611(8164671) in scavenge on r801u23n03 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6612(8164672) in scavenge on r904u01n03 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6613(8164673) in scavenge on r815u23n07 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6614(8164674) in scavenge on r907u34n01 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6615(8164675) in scavenge on r807u08n01 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6616(8164676) in scavenge on r807u08n01 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6617(8164677) in scavenge on r913u24n02 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6618(8164678) in scavenge on r904u10n03 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6619(8164679) in scavenge on r805u02n01 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6620(8164680) in scavenge on r805u02n01 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6621(8164681) in scavenge on r805u11n01 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6622(8164682) in scavenge on r805u14n02 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_591(7571556) in scavenge on r805u14n02 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_910(7571876) in scavenge on r815u29n02 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1060(7572026) in scavenge on r816u29n03 Nov 05 04:14:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_120(7572287) in scavenge on r906u32n01 Nov 05 04:14:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_41(8161834) WEXITSTATUS 0 Nov 05 04:14:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_41(8161834) done Nov 05 04:14:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1610(8153861) WEXITSTATUS 0 Nov 05 04:14:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1610(8153861) done Nov 05 04:14:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3765(8164496) WEXITSTATUS 0 Nov 05 04:14:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3765(8164496) done Nov 05 04:14:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5626(8161746) WEXITSTATUS 1 Nov 05 04:14:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5626(8161746) done Nov 05 04:14:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3775(8164665) WEXITSTATUS 0 Nov 05 04:14:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3775(8164665) done Nov 05 04:14:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6335(8163799) WEXITSTATUS 1 Nov 05 04:14:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6335(8163799) done Nov 05 04:15:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6058(8163081) WEXITSTATUS 1 Nov 05 04:15:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6058(8163081) done Nov 05 04:15:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4683(8156345) WEXITSTATUS 1 Nov 05 04:15:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4683(8156345) done Nov 05 04:15:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3776(8164666) WEXITSTATUS 0 Nov 05 04:15:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3776(8164666) done Nov 05 04:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_38(8161742) WEXITSTATUS 0 Nov 05 04:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_38(8161742) done Nov 05 04:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1671(8156304) WEXITSTATUS 0 Nov 05 04:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1671(8156304) done Nov 05 04:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6460(8164335) WEXITSTATUS 1 Nov 05 04:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6460(8164335) done Nov 05 04:15:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6388(8164099) WEXITSTATUS 1 Nov 05 04:15:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6388(8164099) done Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:15:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6334(8163798) WEXITSTATUS 1 Nov 05 04:15:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6334(8163798) done Nov 05 04:15:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1606(8153242) WEXITSTATUS 0 Nov 05 04:15:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1606(8153242) done Nov 05 04:15:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6243(8163449) WEXITSTATUS 1 Nov 05 04:15:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6243(8163449) done Nov 05 04:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6578(8164615) WEXITSTATUS 1 Nov 05 04:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6578(8164615) done Nov 05 04:16:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5831(8162798) WEXITSTATUS 1 Nov 05 04:16:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5831(8162798) done Nov 05 04:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4996(8158275) WEXITSTATUS 1 Nov 05 04:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4996(8158275) done Nov 05 04:16:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159217 WEXITSTATUS 0 Nov 05 04:16:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159217 done Nov 05 04:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8152284 WEXITSTATUS 0 Nov 05 04:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8152284 done Nov 05 04:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6622(8164682) WEXITSTATUS 1 Nov 05 04:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6622(8164682) done Nov 05 04:17:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3778(8164683) in gpu on r808u14n01 Nov 05 04:17:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3779(8164684) in gpu on r808u14n01 Nov 05 04:17:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3780(8164685) in gpu on r903u34n01 Nov 05 04:17:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2144(8164686) in day on r815u15n08 Nov 05 04:17:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2145(8164687) in day on r816u15n05 Nov 05 04:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1604(8153022) WEXITSTATUS 0 Nov 05 04:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1604(8153022) done Nov 05 04:17:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:17:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6112(8163150) WEXITSTATUS 1 Nov 05 04:17:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6112(8163150) done Nov 05 04:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1593(8152388) WEXITSTATUS 0 Nov 05 04:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1593(8152388) done Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6623(8164688) in scavenge on r802u02n03 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6624(8164689) in scavenge on r802u09n02 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6625(8164690) in scavenge on r816u23n10 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6626(8164691) in scavenge on r816u29n07 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6627(8164692) in scavenge on r806u09n04 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6628(8164693) in scavenge on r806u09n04 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6629(8164694) in scavenge on r904u04n02 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6630(8164695) in scavenge on r904u04n02 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6631(8164696) in scavenge on r805u06n04 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6632(8164697) in scavenge on r805u14n02 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6633(8164698) in scavenge on r913u05n03 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6634(8164699) in scavenge on r913u24n01 Nov 05 04:17:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6635(8164700) in scavenge on r909u04n04 Nov 05 04:17:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6176(8163262) WEXITSTATUS 1 Nov 05 04:17:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6176(8163262) done Nov 05 04:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6363(8163954) WEXITSTATUS 1 Nov 05 04:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6363(8163954) done Nov 05 04:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6593(8164635) WEXITSTATUS 1 Nov 05 04:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6593(8164635) done Nov 05 04:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1677(8156310) WEXITSTATUS 0 Nov 05 04:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1677(8156310) done Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1311(8135376) WEXITSTATUS 0 Nov 05 04:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1311(8135376) done Nov 05 04:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3780(8164685) WEXITSTATUS 0 Nov 05 04:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3780(8164685) done Nov 05 04:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1577(8150820) WEXITSTATUS 0 Nov 05 04:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1577(8150820) done Nov 05 04:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5996(8163001) WEXITSTATUS 1 Nov 05 04:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5996(8163001) done Nov 05 04:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2465(8152623) WEXITSTATUS 0 Nov 05 04:18:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2465(8152623) done Nov 05 04:18:32 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2122(8151798) WEXITSTATUS 0 Nov 05 04:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2122(8151798) done Nov 05 04:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1615(8154191) WEXITSTATUS 0 Nov 05 04:18:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1615(8154191) done Nov 05 04:18:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164701 InitPrio=17389 usec=4711 Nov 05 04:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5925(8162920) WEXITSTATUS 1 Nov 05 04:18:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5925(8162920) done Nov 05 04:18:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5929(8162924) WEXITSTATUS 1 Nov 05 04:18:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5929(8162924) done Nov 05 04:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_892(8128054) WEXITSTATUS 0 Nov 05 04:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_892(8128054) done Nov 05 04:18:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8153545 uid 17375 Nov 05 04:19:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159219 WEXITSTATUS 0 Nov 05 04:19:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159219 done Nov 05 04:19:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164702 InitPrio=17389 usec=5578 Nov 05 04:19:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3779(8164684) WEXITSTATUS 0 Nov 05 04:19:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3779(8164684) done Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:19:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6245(8163451) WEXITSTATUS 1 Nov 05 04:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6245(8163451) done Nov 05 04:19:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8164701 in gpu on r907u34n01 Nov 05 04:19:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8164702 in gpu on r806u35n01 Nov 05 04:19:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3781(8164703) in gpu on r808u14n01 Nov 05 04:19:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3782(8164704) in gpu on r903u34n01 Nov 05 04:19:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2146(8164705) in day on r918u01n01 Nov 05 04:19:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2147(8164706) in day on r919u36n04 Nov 05 04:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2148(8164707) in day on r815u09n02 Nov 05 04:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2149(8164708) in day on r815u09n05 Nov 05 04:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2150(8164709) in day on r815u09n07 Nov 05 04:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2151(8164710) in day on r815u09n07 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6636(8164711) in scavenge on r802u32n04 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6637(8164712) in scavenge on r815u23n12 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6638(8164713) in scavenge on r816u29n10 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6639(8164714) in scavenge on r816u29n11 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6640(8164715) in scavenge on r808u08n01 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6641(8164716) in scavenge on r805u14n03 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6642(8164717) in scavenge on r913u28n04 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_405(7572573) in scavenge on r815u09n07 Nov 05 04:20:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_804(8115073) in scavenge on r815u09n07 Nov 05 04:20:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6405(8164178) WEXITSTATUS 1 Nov 05 04:20:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6405(8164178) done Nov 05 04:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159220 WEXITSTATUS 0 Nov 05 04:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159220 done Nov 05 04:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3782(8164704) WEXITSTATUS 0 Nov 05 04:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3782(8164704) done Nov 05 04:20:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5921(8162913) WEXITSTATUS 1 Nov 05 04:20:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5921(8162913) done Nov 05 04:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6617(8164677) WEXITSTATUS 1 Nov 05 04:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6617(8164677) done Nov 05 04:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150824 WEXITSTATUS 0 Nov 05 04:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150824 done Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:21:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1657(8156242) WEXITSTATUS 0 Nov 05 04:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1657(8156242) done Nov 05 04:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159218 WEXITSTATUS 0 Nov 05 04:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159218 done Nov 05 04:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6130(8163171) WEXITSTATUS 1 Nov 05 04:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6130(8163171) done Nov 05 04:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6399(8164140) WEXITSTATUS 1 Nov 05 04:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6399(8164140) done Nov 05 04:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159194 WEXITSTATUS 0 Nov 05 04:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159194 done Nov 05 04:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6127(8163168) WEXITSTATUS 1 Nov 05 04:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6127(8163168) done Nov 05 04:22:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3783(8164718) in gpu on r903u34n01 Nov 05 04:22:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2152(8164719) in day on r815u03n05 Nov 05 04:22:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6643(8164720) in scavenge on r802u02n03 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6644(8164721) in scavenge on r805u23n04 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6645(8164722) in scavenge on r913u24n01 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6646(8164723) in scavenge on r913u24n02 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6647(8164724) in scavenge on r909u01n03 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6648(8164725) in scavenge on r909u07n04 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1126(8115395) in scavenge on r816u03n06 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1037(7570699) in scavenge on r805u32n01 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1097(7570759) in scavenge on r805u32n01 Nov 05 04:22:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1098(7570760) in scavenge on r805u32n01 Nov 05 04:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6591(8164633) WEXITSTATUS 1 Nov 05 04:22:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6591(8164633) done Nov 05 04:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6630(8164695) WEXITSTATUS 1 Nov 05 04:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6630(8164695) done Nov 05 04:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150823 WEXITSTATUS 0 Nov 05 04:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150823 done Nov 05 04:23:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_48(8135747) WEXITSTATUS 0 Nov 05 04:23:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_48(8135747) done Nov 05 04:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1576(8150819) WEXITSTATUS 0 Nov 05 04:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1576(8150819) done Nov 05 04:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6022(8163032) WEXITSTATUS 1 Nov 05 04:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6022(8163032) done Nov 05 04:23:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5562(8161216) WEXITSTATUS 1 Nov 05 04:23:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5562(8161216) done Nov 05 04:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3781(8164703) WEXITSTATUS 0 Nov 05 04:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3781(8164703) done Nov 05 04:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1782(8158039) WEXITSTATUS 0 Nov 05 04:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1782(8158039) done Nov 05 04:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6407(8164180) WEXITSTATUS 1 Nov 05 04:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6407(8164180) done Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5158(8158512) WEXITSTATUS 1 Nov 05 04:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5158(8158512) done Nov 05 04:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154648 WEXITSTATUS 0 Nov 05 04:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154648 done Nov 05 04:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6525(8164463) WEXITSTATUS 1 Nov 05 04:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6525(8164463) done Nov 05 04:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1575(8150818) WEXITSTATUS 0 Nov 05 04:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1575(8150818) done Nov 05 04:24:04 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1525(8145081) WEXITSTATUS 0 Nov 05 04:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1525(8145081) done Nov 05 04:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5148(8158495) WEXITSTATUS 1 Nov 05 04:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5148(8158495) done Nov 05 04:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1520(8144000) WEXITSTATUS 0 Nov 05 04:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1520(8144000) done Nov 05 04:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6069(8163092) WEXITSTATUS 1 Nov 05 04:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6069(8163092) done Nov 05 04:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1578(8150821) WEXITSTATUS 0 Nov 05 04:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1578(8150821) done Nov 05 04:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1559(8148466) WEXITSTATUS 0 Nov 05 04:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1559(8148466) done Nov 05 04:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162642 WEXITSTATUS 0 Nov 05 04:24:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162642 done Nov 05 04:24:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6309(8163705) WEXITSTATUS 1 Nov 05 04:24:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6309(8163705) done Nov 05 04:24:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150808 WEXITSTATUS 0 Nov 05 04:24:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8150808 done Nov 05 04:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159193 WEXITSTATUS 0 Nov 05 04:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159193 done Nov 05 04:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6453(8164307) WEXITSTATUS 1 Nov 05 04:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6453(8164307) done Nov 05 04:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6235(8163416) WEXITSTATUS 1 Nov 05 04:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6235(8163416) done Nov 05 04:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162683 WEXITSTATUS 0 Nov 05 04:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162683 done Nov 05 04:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6505(8164434) WEXITSTATUS 1 Nov 05 04:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6505(8164434) done Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3784(8164726) in gpu on r808u14n01 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154711 in day on r806u09n03 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2153(8164727) in day on r909u07n02 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_714(7571679) WEXITSTATUS 0 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_714(7571679) done Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1058(7573227) WEXITSTATUS 0 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1058(7573227) done Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_910(7571876) WEXITSTATUS 0 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_910(7571876) done Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_213(7572381) WEXITSTATUS 0 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_213(7572381) done Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_401(7572569) WEXITSTATUS 0 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_401(7572569) done Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_23(8114289) WEXITSTATUS 0 Nov 05 04:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_23(8114289) done Nov 05 04:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_419(7572587) WEXITSTATUS 0 Nov 05 04:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_419(7572587) done Nov 05 04:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_322(8114589) WEXITSTATUS 0 Nov 05 04:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_322(8114589) done Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6649(8164728) in scavenge on r907u01n02 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6650(8164729) in scavenge on r916u01n01 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6651(8164730) in scavenge on r802u06n02 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6652(8164731) in scavenge on r802u11n01 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6653(8164732) in scavenge on r816u23n12 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6654(8164733) in scavenge on r807u03n02 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6655(8164734) in scavenge on r904u04n02 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6656(8164735) in scavenge on r912u32n03 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6657(8164736) in scavenge on r913u01n02 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6658(8164737) in scavenge on r913u24n04 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6659(8164738) in scavenge on r909u07n04 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6660(8164739) in scavenge on r909u10n01 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6661(8164740) in scavenge on r909u10n01 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_20(7570983) in scavenge on r919u36n04 Nov 05 04:25:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_556(7571521) in scavenge on r908u24n01 Nov 05 04:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162663 WEXITSTATUS 0 Nov 05 04:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162663 done Nov 05 04:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159216 WEXITSTATUS 0 Nov 05 04:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159216 done Nov 05 04:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159191 WEXITSTATUS 0 Nov 05 04:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159191 done Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:25:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:26:03 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4537(8155942) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:26:03 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4542(8155947) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:26:03 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4952(8158197) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:26:03 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5838(8162805) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:26:03 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5839(8162806) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:26:03 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5895(8162871) has been requeued to reclaim resources for JobId=Invalid Nov 05 04:26:04 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5838(8162805) Nov 05 04:26:05 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4537(8155942) Nov 05 04:26:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4952(8158197) Nov 05 04:26:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5895(8162871) Nov 05 04:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5705(8162418) WEXITSTATUS 1 Nov 05 04:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5705(8162418) done Nov 05 04:26:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4542(8155947) Nov 05 04:26:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5839(8162806) Nov 05 04:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_48(8162161) WEXITSTATUS 0 Nov 05 04:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_48(8162161) done Nov 05 04:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5948(8162944) WEXITSTATUS 1 Nov 05 04:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5948(8162944) done Nov 05 04:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6633(8164698) WEXITSTATUS 1 Nov 05 04:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6633(8164698) done Nov 05 04:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1633(8155442) WEXITSTATUS 0 Nov 05 04:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1633(8155442) done Nov 05 04:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1573(8150816) WEXITSTATUS 0 Nov 05 04:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1573(8150816) done Nov 05 04:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159195 WEXITSTATUS 0 Nov 05 04:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159195 done Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8158167 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 #CPUs=288 Partition=day Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2154(8164741) NodeList=r816u09n03 #CPUs=1 Partition=day Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2155(8164742) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2156(8164743) NodeList=r815u03n03 #CPUs=1 Partition=day Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2157(8164744) NodeList=r815u03n03 #CPUs=1 Partition=day Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2158(8164745) NodeList=r815u09n01 #CPUs=1 Partition=day Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2159(8164746) NodeList=r815u09n08 #CPUs=1 Partition=day Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6584(8164624) WEXITSTATUS 1 Nov 05 04:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6584(8164624) done Nov 05 04:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6576(8164613) WEXITSTATUS 1 Nov 05 04:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6576(8164613) done Nov 05 04:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2329(8126296) WEXITSTATUS 1 Nov 05 04:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2329(8126296) done Nov 05 04:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6654(8164733) WEXITSTATUS 1 Nov 05 04:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6654(8164733) done Nov 05 04:27:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6080(8163107) WEXITSTATUS 1 Nov 05 04:27:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6080(8163107) done Nov 05 04:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6612(8164672) WEXITSTATUS 1 Nov 05 04:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6612(8164672) done Nov 05 04:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6454(8164329) WEXITSTATUS 1 Nov 05 04:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6454(8164329) done Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:27:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3778(8164683) WEXITSTATUS 0 Nov 05 04:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3778(8164683) done Nov 05 04:27:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3785(8164747) in gpu on r808u14n01 Nov 05 04:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6040(8163062) WEXITSTATUS 1 Nov 05 04:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6040(8163062) done Nov 05 04:27:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_624(8164288) WEXITSTATUS 0 Nov 05 04:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_624(8164288) done Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6662(8164748) in scavenge on r903u01n01 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6663(8164749) in scavenge on r802u11n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6664(8164750) in scavenge on r802u30n04 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6665(8164751) in scavenge on r904u01n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6666(8164752) in scavenge on r904u01n03 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6667(8164753) in scavenge on r906u39n01 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6668(8164754) in scavenge on r807u03n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6669(8164755) in scavenge on r913u05n03 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6670(8164756) in scavenge on r904u10n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6671(8164757) in scavenge on r909u13n01 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6672(8164758) in scavenge on r909u13n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6673(8164759) in scavenge on r909u13n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6674(8164760) in scavenge on r909u29n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6675(8164761) in scavenge on r918u05n01 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6676(8164762) in scavenge on r918u05n03 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6677(8164763) in scavenge on r918u09n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6678(8164764) in scavenge on r918u09n04 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_578(7571543) in scavenge on r816u03n01 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_644(7571609) in scavenge on r816u29n03 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_853(7571819) in scavenge on r815u23n05 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_856(7571822) in scavenge on r815u35n01 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_882(7571848) in scavenge on r815u03n02 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_103(7572270) in scavenge on r815u03n03 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_106(7572273) in scavenge on r815u09n01 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_216(7572384) in scavenge on r815u09n08 Nov 05 04:28:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_256(7572424) in scavenge on r815u15n08 Nov 05 04:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1711(8156973) WEXITSTATUS 0 Nov 05 04:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1711(8156973) done Nov 05 04:28:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2160(8164765) in scavenge on r911u05n02 Nov 05 04:28:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2161(8164766) in scavenge on r816u35n07 Nov 05 04:28:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2162(8164767) in scavenge on r906u32n01 Nov 05 04:28:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2163(8164768) in scavenge on r906u32n01 Nov 05 04:28:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2164(8164769) in scavenge on r807u37n01 Nov 05 04:28:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2165(8164770) in scavenge on r807u37n01 Nov 05 04:28:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2166(8164771) in scavenge on r807u37n01 Nov 05 04:28:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2167(8164772) in scavenge on r807u37n01 Nov 05 04:28:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2168(8164773) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 04:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3784(8164726) WEXITSTATUS 0 Nov 05 04:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3784(8164726) done Nov 05 04:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_49(8158658) WEXITSTATUS 0 Nov 05 04:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_49(8158658) done Nov 05 04:29:04 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:29:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1648(8156079) WEXITSTATUS 0 Nov 05 04:29:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1648(8156079) done Nov 05 04:29:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6355(8163899) WEXITSTATUS 1 Nov 05 04:29:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6355(8163899) done Nov 05 04:29:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6270(8163552) WEXITSTATUS 1 Nov 05 04:29:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6270(8163552) done Nov 05 04:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1634(8155684) WEXITSTATUS 0 Nov 05 04:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1634(8155684) done Nov 05 04:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2739(8128191) WEXITSTATUS 1 Nov 05 04:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2739(8128191) done Nov 05 04:29:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2169(8164774) NodeList=r815u35n03 #CPUs=1 Partition=day Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:29:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:29:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5729(8162562) WEXITSTATUS 1 Nov 05 04:29:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5729(8162562) done Nov 05 04:30:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5909(8162901) WEXITSTATUS 1 Nov 05 04:30:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5909(8162901) done Nov 05 04:30:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162652 WEXITSTATUS 0 Nov 05 04:30:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162652 done Nov 05 04:30:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 04:30:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 04:30:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2170(8164775) NodeList=r815u09n04 #CPUs=1 Partition=day Nov 05 04:30:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3786(8164776) in gpu on r808u14n01 Nov 05 04:30:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1707(8156852) WEXITSTATUS 0 Nov 05 04:30:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1707(8156852) done Nov 05 04:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6679(8164777) in scavenge on r917u01n02 Nov 05 04:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6680(8164778) in scavenge on r815u23n09 Nov 05 04:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6681(8164779) in scavenge on r816u29n07 Nov 05 04:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6682(8164780) in scavenge on r816u35n11 Nov 05 04:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6683(8164781) in scavenge on r906u41n02 Nov 05 04:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_296(7572464) in scavenge on r815u09n04 Nov 05 04:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_682(7572851) in scavenge on r815u09n05 Nov 05 04:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_692(7572861) in scavenge on r815u09n05 Nov 05 04:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6189(8163311) WEXITSTATUS 1 Nov 05 04:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6189(8163311) done Nov 05 04:30:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2171(8164782) in scavenge on r816u23n04 Nov 05 04:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6639(8164714) WEXITSTATUS 1 Nov 05 04:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6639(8164714) done Nov 05 04:30:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164336 WEXITSTATUS 0 Nov 05 04:30:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164336 done Nov 05 04:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 04:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 04:30:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 04:31:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6683(8164781) WEXITSTATUS 1 Nov 05 04:31:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6683(8164781) done Nov 05 04:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6679(8164777) WEXITSTATUS 1 Nov 05 04:31:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6679(8164777) done Nov 05 04:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3785(8164747) WEXITSTATUS 0 Nov 05 04:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3785(8164747) done Nov 05 04:31:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162692 WEXITSTATUS 0 Nov 05 04:31:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162692 done Nov 05 04:31:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2172(8164783) NodeList=r805u06n01 #CPUs=1 Partition=day Nov 05 04:31:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2173(8164784) NodeList=r815u35n06 #CPUs=1 Partition=day Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:31:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6395(8164136) WEXITSTATUS 1 Nov 05 04:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6395(8164136) done Nov 05 04:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5719(8162521) WEXITSTATUS 1 Nov 05 04:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5719(8162521) done Nov 05 04:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6658(8164737) WEXITSTATUS 1 Nov 05 04:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6658(8164737) done Nov 05 04:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4984(8158257) WEXITSTATUS 1 Nov 05 04:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4984(8158257) done Nov 05 04:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1630(8155207) WEXITSTATUS 0 Nov 05 04:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1630(8155207) done Nov 05 04:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2503(8152661) WEXITSTATUS 0 Nov 05 04:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2503(8152661) done Nov 05 04:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6493(8164417) WEXITSTATUS 1 Nov 05 04:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6493(8164417) done Nov 05 04:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_43(8161934) WEXITSTATUS 0 Nov 05 04:32:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_43(8161934) done Nov 05 04:32:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 04:32:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 04:32:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 04:32:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6602(8164657) WEXITSTATUS 1 Nov 05 04:32:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6602(8164657) done Nov 05 04:32:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6011(8163016) WEXITSTATUS 1 Nov 05 04:32:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6011(8163016) done Nov 05 04:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6471(8164376) WEXITSTATUS 1 Nov 05 04:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6471(8164376) done Nov 05 04:33:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8158639: assoc 9349 TRES gres/gpu:v100 grp_used_tres_run_secs underflow, tried to remove 120 seconds when only 110 remained. Nov 05 04:33:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8158639: assoc 5491 TRES gres/gpu:v100 grp_used_tres_run_secs underflow, tried to remove 120 seconds when only 110 remained. Nov 05 04:33:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8158639: assoc 9349 TRES gres/gpu:v100 grp_used_tres_run_secs underflow, tried to remove 120 seconds when only 110 remained. Nov 05 04:33:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8158639: assoc 5491 TRES gres/gpu:v100 grp_used_tres_run_secs underflow, tried to remove 120 seconds when only 110 remained. Nov 05 04:33:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3787(8164785) in gpu on r808u14n01 Nov 05 04:33:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6684(8164786) in scavenge on r801u30n04 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6685(8164787) in scavenge on r908u33n03 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6686(8164788) in scavenge on r801u14n01 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6687(8164789) in scavenge on r916u04n02 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6688(8164790) in scavenge on r916u04n01 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6689(8164791) in scavenge on r917u01n02 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6690(8164792) in scavenge on r816u29n11 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6691(8164793) in scavenge on r805u02n02 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6692(8164794) in scavenge on r805u02n02 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6693(8164795) in scavenge on r807u11n01 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6694(8164796) in scavenge on r906u41n02 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6695(8164797) in scavenge on r909u13n04 Nov 05 04:33:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_46(8114312) in scavenge on r913u24n04 Nov 05 04:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5239(8158646) WEXITSTATUS 1 Nov 05 04:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5239(8158646) done Nov 05 04:33:34 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2410(8126551) Nov 05 04:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6537(8164487) WEXITSTATUS 1 Nov 05 04:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6537(8164487) done Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:33:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5972(8162972) WEXITSTATUS 1 Nov 05 04:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5972(8162972) done Nov 05 04:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133373 WEXITSTATUS 0 Nov 05 04:33:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133373 done Nov 05 04:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5881(8162857) WEXITSTATUS 1 Nov 05 04:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5881(8162857) done Nov 05 04:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5361(8159171) WEXITSTATUS 1 Nov 05 04:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5361(8159171) done Nov 05 04:34:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2633(8152900) WEXITSTATUS 0 Nov 05 04:34:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2633(8152900) done Nov 05 04:34:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5867(8162839) WEXITSTATUS 1 Nov 05 04:34:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5867(8162839) done Nov 05 04:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3787(8164785) WEXITSTATUS 0 Nov 05 04:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3787(8164785) done Nov 05 04:34:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8158639: assoc 9349 TRES gres/gpu:v100 grp_used_tres_run_secs underflow, tried to remove 110 seconds when only 0 remained. Nov 05 04:34:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8158639: assoc 5491 TRES gres/gpu:v100 grp_used_tres_run_secs underflow, tried to remove 110 seconds when only 0 remained. Nov 05 04:34:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8158639: assoc 9349 TRES gres/gpu:v100 grp_used_tres_run_secs underflow, tried to remove 110 seconds when only 0 remained. Nov 05 04:34:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8158639: assoc 5491 TRES gres/gpu:v100 grp_used_tres_run_secs underflow, tried to remove 110 seconds when only 0 remained. Nov 05 04:34:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5613(8161714) WEXITSTATUS 1 Nov 05 04:34:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5613(8161714) done Nov 05 04:34:33 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158639 Nov 05 04:34:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:34:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_991(8128377) WEXITSTATUS 0 Nov 05 04:34:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_991(8128377) done Nov 05 04:34:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2454(8152612) WEXITSTATUS 0 Nov 05 04:34:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2454(8152612) done Nov 05 04:34:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6103(8163141) WEXITSTATUS 1 Nov 05 04:34:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6103(8163141) done Nov 05 04:34:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5865(8162837) WEXITSTATUS 1 Nov 05 04:34:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5865(8162837) done Nov 05 04:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1653(8156201) WEXITSTATUS 0 Nov 05 04:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1653(8156201) done Nov 05 04:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1551(8146920) WEXITSTATUS 0 Nov 05 04:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1551(8146920) done Nov 05 04:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1755(8157797) WEXITSTATUS 0 Nov 05 04:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1755(8157797) done Nov 05 04:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1543(8146146) WEXITSTATUS 0 Nov 05 04:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1543(8146146) done Nov 05 04:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2037(8125815) WEXITSTATUS 1 Nov 05 04:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2037(8125815) done Nov 05 04:35:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6016(8163026) WEXITSTATUS 1 Nov 05 04:35:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6016(8163026) done Nov 05 04:35:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6411(8164184) WEXITSTATUS 1 Nov 05 04:35:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6411(8164184) done Nov 05 04:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6210(8163346) WEXITSTATUS 1 Nov 05 04:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6210(8163346) done Nov 05 04:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2174(8164798) NodeList=r914u05n04 #CPUs=1 Partition=day Nov 05 04:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2175(8164799) NodeList=r815u35n01 #CPUs=1 Partition=day Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:35:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:35:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1550(8146919) WEXITSTATUS 0 Nov 05 04:35:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1550(8146919) done Nov 05 04:36:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3788(8164800) in gpu on r808u14n01 Nov 05 04:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5159(8158513) WEXITSTATUS 1 Nov 05 04:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5159(8158513) done Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6696(8164801) in scavenge on r908u33n04 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6697(8164802) in scavenge on r903u07n01 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6698(8164803) in scavenge on r802u04n01 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6699(8164804) in scavenge on r802u11n02 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6700(8164805) in scavenge on r916u01n01 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6701(8164806) in scavenge on r917u07n04 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6702(8164807) in scavenge on r815u15n10 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6703(8164808) in scavenge on r816u09n10 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6704(8164809) in scavenge on r816u23n08 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6705(8164810) in scavenge on r816u23n08 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6706(8164811) in scavenge on r902u07n01 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6707(8164812) in scavenge on r806u09n02 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6708(8164813) in scavenge on r912u28n02 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6709(8164814) in scavenge on r918u01n03 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6710(8164815) in scavenge on r918u05n02 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6641(8164716) WEXITSTATUS 1 Nov 05 04:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6641(8164716) done Nov 05 04:36:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2176(8164816) in scavenge on r912u13n01 Nov 05 04:36:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2177(8164817) in scavenge on r912u13n01 Nov 05 04:36:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2178(8164818) in scavenge on r806u31n01 Nov 05 04:36:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5637(8161839) WEXITSTATUS 1 Nov 05 04:36:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5637(8161839) done Nov 05 04:36:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6409(8164182) WEXITSTATUS 1 Nov 05 04:36:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6409(8164182) done Nov 05 04:36:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6419(8164226) WEXITSTATUS 1 Nov 05 04:36:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6419(8164226) done Nov 05 04:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5924(8162919) WEXITSTATUS 1 Nov 05 04:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5924(8162919) done Nov 05 04:36:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6469(8164374) WEXITSTATUS 1 Nov 05 04:36:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6469(8164374) done Nov 05 04:37:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3178(8131800) WEXITSTATUS 1 Nov 05 04:37:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3178(8131800) done Nov 05 04:37:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6446(8164276) WEXITSTATUS 1 Nov 05 04:37:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6446(8164276) done Nov 05 04:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6337(8163840) WEXITSTATUS 1 Nov 05 04:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6337(8163840) done Nov 05 04:37:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144152 NodeList=r802u25n03,r807u01n02 #CPUs=72 Partition=scavenge Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6485(8164401) WEXITSTATUS 1 Nov 05 04:37:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6485(8164401) done Nov 05 04:37:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1649(8156087) WEXITSTATUS 0 Nov 05 04:37:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1649(8156087) done Nov 05 04:38:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6322(8163752) WEXITSTATUS 1 Nov 05 04:38:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6322(8163752) done Nov 05 04:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1613(8154046) WEXITSTATUS 0 Nov 05 04:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1613(8154046) done Nov 05 04:38:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2179(8164819) NodeList=r815u09n04 #CPUs=1 Partition=day Nov 05 04:38:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2180(8164820) NodeList=r816u03n08 #CPUs=1 Partition=day Nov 05 04:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6696(8164801) WEXITSTATUS 1 Nov 05 04:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6696(8164801) done Nov 05 04:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3786(8164776) WEXITSTATUS 0 Nov 05 04:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3786(8164776) done Nov 05 04:38:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3789(8164821) in gpu on r808u14n01 Nov 05 04:38:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6711(8164822) in scavenge on r908u33n04 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6712(8164823) in scavenge on r907u09n03 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6713(8164824) in scavenge on r908u35n01 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6714(8164825) in scavenge on r802u11n03 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6715(8164826) in scavenge on r903u04n03 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6716(8164827) in scavenge on r815u23n12 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6717(8164828) in scavenge on r816u15n10 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6718(8164829) in scavenge on r909u01n04 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6719(8164830) in scavenge on r913u13n03 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6720(8164831) in scavenge on r805u14n03 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6721(8164832) in scavenge on r805u14n04 Nov 05 04:38:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6722(8164833) in scavenge on r918u09n04 Nov 05 04:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1567(8149341) WEXITSTATUS 0 Nov 05 04:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1567(8149341) done Nov 05 04:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6669(8164755) WEXITSTATUS 1 Nov 05 04:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6669(8164755) done Nov 05 04:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6166(8163244) WEXITSTATUS 1 Nov 05 04:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6166(8163244) done Nov 05 04:39:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162649 WEXITSTATUS 0 Nov 05 04:39:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162649 done Nov 05 04:39:28 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r913u28n01 reason set to: NHC: check_cmd_status: 142 (expected 0) returned by "test -e /vast/palmer/home.grace/.yalehpc". Nov 05 04:39:28 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r913u28n01 state set to DRAINING Nov 05 04:39:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 04:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6671(8164757) WEXITSTATUS 1 Nov 05 04:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6671(8164757) done Nov 05 04:39:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2181(8164834) NodeList=r918u05n04 #CPUs=1 Partition=day Nov 05 04:39:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2182(8164835) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 04:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6372(8164000) WEXITSTATUS 1 Nov 05 04:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6372(8164000) done Nov 05 04:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4578(8156091) WEXITSTATUS 1 Nov 05 04:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4578(8156091) done Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:39:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5573(8161227) WEXITSTATUS 1 Nov 05 04:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5573(8161227) done Nov 05 04:39:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1801(8158159) WEXITSTATUS 0 Nov 05 04:39:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1801(8158159) done Nov 05 04:40:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:40:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6330(8163794) WEXITSTATUS 1 Nov 05 04:40:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6330(8163794) done Nov 05 04:40:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5703(8162416) WEXITSTATUS 1 Nov 05 04:40:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5703(8162416) done Nov 05 04:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162693 WEXITSTATUS 0 Nov 05 04:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162693 done Nov 05 04:40:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2183(8164836) NodeList=r918u32n01 #CPUs=1 Partition=day Nov 05 04:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6222(8163374) WEXITSTATUS 1 Nov 05 04:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6222(8163374) done Nov 05 04:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3788(8164800) WEXITSTATUS 0 Nov 05 04:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3788(8164800) done Nov 05 04:40:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159192 WEXITSTATUS 0 Nov 05 04:40:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159192 done Nov 05 04:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4894(8158027) WEXITSTATUS 1 Nov 05 04:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4894(8158027) done Nov 05 04:40:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6271(8163553) WEXITSTATUS 1 Nov 05 04:40:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6271(8163553) done Nov 05 04:41:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162433 WEXITSTATUS 0 Nov 05 04:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162433 done Nov 05 04:41:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6296(8163649) WEXITSTATUS 1 Nov 05 04:41:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6296(8163649) done Nov 05 04:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6675(8164761) WEXITSTATUS 1 Nov 05 04:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6675(8164761) done Nov 05 04:41:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3790(8164837) in gpu on r808u14n01 Nov 05 04:41:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6723(8164838) in scavenge on r906u01n02 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6724(8164839) in scavenge on r908u39n03 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6725(8164840) in scavenge on r802u04n03 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6726(8164841) in scavenge on r816u15n11 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6727(8164842) in scavenge on r808u11n01 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6728(8164843) in scavenge on r906u41n02 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6729(8164844) in scavenge on r913u05n03 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6730(8164845) in scavenge on r805u09n02 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6731(8164846) in scavenge on r805u14n03 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6732(8164847) in scavenge on r909u07n04 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6733(8164848) in scavenge on r909u13n01 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6734(8164849) in scavenge on r918u05n01 Nov 05 04:41:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6735(8164850) in scavenge on r918u32n02 Nov 05 04:41:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 04:41:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2184(8164851) in scavenge on r904u13n01 Nov 05 04:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3790(8164837) WEXITSTATUS 0 Nov 05 04:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3790(8164837) done Nov 05 04:41:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6104(8163142) WEXITSTATUS 1 Nov 05 04:41:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6104(8163142) done Nov 05 04:41:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4236(8152695) WEXITSTATUS 1 Nov 05 04:41:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4236(8152695) done Nov 05 04:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5979(8162979) WEXITSTATUS 1 Nov 05 04:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5979(8162979) done Nov 05 04:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6663(8164749) WEXITSTATUS 1 Nov 05 04:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6663(8164749) done Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:41:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1682(8156315) WEXITSTATUS 0 Nov 05 04:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1682(8156315) done Nov 05 04:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1646(8155952) WEXITSTATUS 0 Nov 05 04:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1646(8155952) done Nov 05 04:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_1(8162737) WEXITSTATUS 0 Nov 05 04:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_1(8162737) done Nov 05 04:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_2(8162738) WEXITSTATUS 0 Nov 05 04:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_2(8162738) done Nov 05 04:42:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2185(8164852) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 04:42:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2186(8164853) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 04:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_44(8161935) WEXITSTATUS 0 Nov 05 04:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_44(8161935) done Nov 05 04:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6482(8164389) WEXITSTATUS 1 Nov 05 04:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6482(8164389) done Nov 05 04:42:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1598(8152801) WEXITSTATUS 0 Nov 05 04:42:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1598(8152801) done Nov 05 04:43:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6547(8164500) WEXITSTATUS 1 Nov 05 04:43:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6547(8164500) done Nov 05 04:43:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6704(8164809) WEXITSTATUS 1 Nov 05 04:43:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6704(8164809) done Nov 05 04:43:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1619(8154565) WEXITSTATUS 0 Nov 05 04:43:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1619(8154565) done Nov 05 04:43:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6719(8164830) WEXITSTATUS 1 Nov 05 04:43:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6719(8164830) done Nov 05 04:43:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2620(8152887) WEXITSTATUS 0 Nov 05 04:43:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2620(8152887) done Nov 05 04:43:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2187(8164854) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 04:43:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2188(8164855) NodeList=r816u03n04 #CPUs=1 Partition=day Nov 05 04:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6432(8164262) WEXITSTATUS 1 Nov 05 04:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6432(8164262) done Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:43:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:44:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3791(8164856) in gpu on r808u14n01 Nov 05 04:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162669 WEXITSTATUS 0 Nov 05 04:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162669 done Nov 05 04:44:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6736(8164857) in scavenge on r908u37n04 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6737(8164858) in scavenge on r802u11n02 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6738(8164859) in scavenge on r802u11n03 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6739(8164860) in scavenge on r815u09n09 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6740(8164861) in scavenge on r815u23n08 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6741(8164862) in scavenge on r816u23n08 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6742(8164863) in scavenge on r805u25n01 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6743(8164864) in scavenge on r805u25n01 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6744(8164865) in scavenge on r909u07n03 Nov 05 04:44:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_478(8114745) in scavenge on r913u13n03 Nov 05 04:44:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_691(8126156) WEXITSTATUS 0 Nov 05 04:44:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_691(8126156) done Nov 05 04:44:27 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r913u28n01 state set to ALLOCATED Nov 05 04:44:28 r912u23n01 slurmctld[163760]: slurmctld: Node r913u28n01 now responding Nov 05 04:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 04:44:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2189(8164866) NodeList=r805u11n02 #CPUs=1 Partition=day Nov 05 04:44:39 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 04:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1549(8146918) WEXITSTATUS 0 Nov 05 04:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1549(8146918) done Nov 05 04:45:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6647(8164724) WEXITSTATUS 1 Nov 05 04:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6647(8164724) done Nov 05 04:45:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2626(8152893) WEXITSTATUS 0 Nov 05 04:45:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2626(8152893) done Nov 05 04:45:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6481(8164388) WEXITSTATUS 1 Nov 05 04:45:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6481(8164388) done Nov 05 04:45:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3732(8164040) WEXITSTATUS 0 Nov 05 04:45:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3732(8164040) done Nov 05 04:45:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162647 WEXITSTATUS 0 Nov 05 04:45:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162647 done Nov 05 04:45:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162662 WEXITSTATUS 0 Nov 05 04:45:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162662 done Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:45:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:46:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_3(8162739) WEXITSTATUS 0 Nov 05 04:46:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_3(8162739) done Nov 05 04:46:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5913(8162905) WEXITSTATUS 1 Nov 05 04:46:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5913(8162905) done Nov 05 04:46:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2190(8164867) NodeList=r815u09n07 #CPUs=1 Partition=day Nov 05 04:46:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2191(8164868) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 04:46:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6598(8164640) WEXITSTATUS 1 Nov 05 04:46:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6598(8164640) done Nov 05 04:46:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3792(8164869) in gpu on r808u17n01 Nov 05 04:46:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6745(8164870) in scavenge on r808u14n01 Nov 05 04:46:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6746(8164871) in scavenge on r805u11n02 Nov 05 04:46:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6747(8164872) in scavenge on r807u11n01 Nov 05 04:46:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6748(8164873) in scavenge on r909u01n03 Nov 05 04:46:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3679(8138255) WEXITSTATUS 1 Nov 05 04:46:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3679(8138255) done Nov 05 04:46:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2192(8164874) in scavenge on r912u13n01 Nov 05 04:46:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2193(8164875) in scavenge on r906u32n01 Nov 05 04:46:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6667(8164753) WEXITSTATUS 1 Nov 05 04:46:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6667(8164753) done Nov 05 04:47:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3791(8164856) WEXITSTATUS 0 Nov 05 04:47:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3791(8164856) done Nov 05 04:47:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1552(8146921) WEXITSTATUS 0 Nov 05 04:47:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1552(8146921) done Nov 05 04:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6441(8164271) WEXITSTATUS 1 Nov 05 04:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6441(8164271) done Nov 05 04:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6339(8163842) WEXITSTATUS 1 Nov 05 04:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6339(8163842) done Nov 05 04:47:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6691(8164793) WEXITSTATUS 1 Nov 05 04:47:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6691(8164793) done Nov 05 04:47:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6015(8163025) WEXITSTATUS 1 Nov 05 04:47:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6015(8163025) done Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:47:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1810(8158351) WEXITSTATUS 0 Nov 05 04:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1810(8158351) done Nov 05 04:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1621(8154567) WEXITSTATUS 0 Nov 05 04:48:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1621(8154567) done Nov 05 04:48:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2194(8164876) NodeList=r815u23n01 #CPUs=1 Partition=day Nov 05 04:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6652(8164731) WEXITSTATUS 1 Nov 05 04:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6652(8164731) done Nov 05 04:48:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6282(8163601) WEXITSTATUS 1 Nov 05 04:48:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6282(8163601) done Nov 05 04:49:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164877 InitPrio=17164 usec=7649 Nov 05 04:49:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8164877 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 04:49:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2427(8127173) Nov 05 04:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159198 WEXITSTATUS 0 Nov 05 04:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159198 done Nov 05 04:49:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6518(8164453) WEXITSTATUS 1 Nov 05 04:49:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6518(8164453) done Nov 05 04:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3792(8164869) WEXITSTATUS 0 Nov 05 04:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3792(8164869) done Nov 05 04:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6090(8163123) WEXITSTATUS 1 Nov 05 04:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6090(8163123) done Nov 05 04:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4047(8147288) WEXITSTATUS 1 Nov 05 04:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4047(8147288) done Nov 05 04:49:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3793(8164878) in gpu on r808u14n01 Nov 05 04:49:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3794(8164879) in gpu on r808u17n01 Nov 05 04:49:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6749(8164880) in scavenge on r801u06n03 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6750(8164881) in scavenge on r903u01n01 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6751(8164882) in scavenge on r802u06n02 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6752(8164883) in scavenge on r802u11n01 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6753(8164884) in scavenge on r906u39n01 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6754(8164885) in scavenge on r805u02n02 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6755(8164886) in scavenge on r806u04n03 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6756(8164887) in scavenge on r914u01n01 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6757(8164888) in scavenge on r909u01n03 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6758(8164889) in scavenge on r909u29n03 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6759(8164890) in scavenge on r918u05n01 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6760(8164891) in scavenge on r918u36n02 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_1009(8116484) in scavenge on r908u37n02 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_317(8114584) in scavenge on r816u35n06 Nov 05 04:49:33 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 04:49:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2195(8164892) in scavenge on r912u13n01 Nov 05 04:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3794(8164879) WEXITSTATUS 0 Nov 05 04:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3794(8164879) done Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:49:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6014(8163024) WEXITSTATUS 1 Nov 05 04:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6014(8163024) done Nov 05 04:50:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1636(8155754) WEXITSTATUS 0 Nov 05 04:50:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1636(8155754) done Nov 05 04:50:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1639(8155797) WEXITSTATUS 0 Nov 05 04:50:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1639(8155797) done Nov 05 04:50:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_992(8128378) WEXITSTATUS 0 Nov 05 04:50:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_992(8128378) done Nov 05 04:50:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:50:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6734(8164849) WEXITSTATUS 1 Nov 05 04:50:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6734(8164849) done Nov 05 04:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159196 WEXITSTATUS 0 Nov 05 04:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159196 done Nov 05 04:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6536(8164481) WEXITSTATUS 1 Nov 05 04:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6536(8164481) done Nov 05 04:51:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1645(8155937) WEXITSTATUS 0 Nov 05 04:51:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1645(8155937) done Nov 05 04:51:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3739(8164169) WEXITSTATUS 0 Nov 05 04:51:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3739(8164169) done Nov 05 04:51:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2196(8164893) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 04:51:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2197(8164894) NodeList=r816u09n08 #CPUs=1 Partition=day Nov 05 04:51:38 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 04:51:40 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158750 Nov 05 04:51:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154653 WEXITSTATUS 0 Nov 05 04:51:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154653 done Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:51:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:52:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3795(8164895) in gpu on r808u17n01 Nov 05 04:52:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3796(8164896) in gpu on r4513u29n01 Nov 05 04:52:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154716 in day on r815u15n03 Nov 05 04:52:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6761(8164897) in scavenge on r912u05n04 Nov 05 04:52:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6762(8164898) in scavenge on r914u05n01 Nov 05 04:52:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6763(8164899) in scavenge on r914u05n01 Nov 05 04:52:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_771(7572940) in scavenge on r815u09n03 Nov 05 04:52:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_893(7571859) in scavenge on r815u09n03 Nov 05 04:52:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_919(7571885) in scavenge on r816u29n03 Nov 05 04:52:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2198(8164900) in scavenge on r815u09n10 Nov 05 04:52:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2199(8164901) in scavenge on r815u29n08 Nov 05 04:52:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2200(8164902) in scavenge on r815u29n11 Nov 05 04:52:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2201(8164903) in scavenge on r807u14n01 Nov 05 04:52:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2202(8164904) in scavenge on r807u14n01 Nov 05 04:52:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2203(8164905) in scavenge on r807u14n01 Nov 05 04:52:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2204(8164906) in scavenge on r906u28n01 Nov 05 04:52:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6523(8164461) WEXITSTATUS 1 Nov 05 04:52:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6523(8164461) done Nov 05 04:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2455(8152613) WEXITSTATUS 0 Nov 05 04:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2455(8152613) done Nov 05 04:52:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1761(8157830) WEXITSTATUS 0 Nov 05 04:52:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1761(8157830) done Nov 05 04:53:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6743(8164864) WEXITSTATUS 1 Nov 05 04:53:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6743(8164864) done Nov 05 04:53:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162656 WEXITSTATUS 0 Nov 05 04:53:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162656 done Nov 05 04:53:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6264(8163546) WEXITSTATUS 1 Nov 05 04:53:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6264(8163546) done Nov 05 04:53:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6477(8164384) WEXITSTATUS 1 Nov 05 04:53:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6477(8164384) done Nov 05 04:53:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6661(8164740) WEXITSTATUS 1 Nov 05 04:53:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6661(8164740) done Nov 05 04:53:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2205(8164907) NodeList=r918u09n01 #CPUs=1 Partition=day Nov 05 04:53:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2206(8164908) NodeList=r816u03n01 #CPUs=1 Partition=day Nov 05 04:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1757(8157799) WEXITSTATUS 0 Nov 05 04:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1757(8157799) done Nov 05 04:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159930 WEXITSTATUS 0 Nov 05 04:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159930 done Nov 05 04:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6742(8164863) WEXITSTATUS 1 Nov 05 04:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6742(8164863) done Nov 05 04:53:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6740(8164861) WEXITSTATUS 1 Nov 05 04:53:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6740(8164861) done Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:53:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3796(8164896) WEXITSTATUS 0 Nov 05 04:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3796(8164896) done Nov 05 04:54:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3795(8164895) WEXITSTATUS 0 Nov 05 04:54:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3795(8164895) done Nov 05 04:54:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6375(8164046) WEXITSTATUS 1 Nov 05 04:54:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6375(8164046) done Nov 05 04:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2207(8164909) NodeList=r815u23n06 #CPUs=1 Partition=day Nov 05 04:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2208(8164910) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 04:54:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3797(8164911) in gpu on r808u17n01 Nov 05 04:54:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3798(8164912) in gpu on r4513u29n01 Nov 05 04:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6451(8164305) WEXITSTATUS 1 Nov 05 04:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6451(8164305) done Nov 05 04:54:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6764(8164913) in scavenge on r802u30n01 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6765(8164914) in scavenge on r815u09n12 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6766(8164915) in scavenge on r815u23n08 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6767(8164916) in scavenge on r903u34n01 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6768(8164917) in scavenge on r805u25n01 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6769(8164918) in scavenge on r805u25n01 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6770(8164919) in scavenge on r909u10n01 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6771(8164920) in scavenge on r909u13n02 Nov 05 04:54:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6772(8164921) in scavenge on r918u01n03 Nov 05 04:55:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3793(8164878) WEXITSTATUS 0 Nov 05 04:55:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3793(8164878) done Nov 05 04:55:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5007(8158291) WEXITSTATUS 1 Nov 05 04:55:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5007(8158291) done Nov 05 04:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2494(8152652) WEXITSTATUS 0 Nov 05 04:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2494(8152652) done Nov 05 04:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_46(8162023) WEXITSTATUS 0 Nov 05 04:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_46(8162023) done Nov 05 04:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6030(8163046) WEXITSTATUS 1 Nov 05 04:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6030(8163046) done Nov 05 04:55:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3798(8164912) WEXITSTATUS 0 Nov 05 04:55:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3798(8164912) done Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:56:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:56:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8164922 InitPrio=20041 usec=4384 Nov 05 04:56:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158167 WEXITSTATUS 0 Nov 05 04:56:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158167 done Nov 05 04:56:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158861 Nov 05 04:56:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 04:56:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6274(8163556) WEXITSTATUS 1 Nov 05 04:56:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6274(8163556) done Nov 05 04:56:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6761(8164897) WEXITSTATUS 1 Nov 05 04:56:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6761(8164897) done Nov 05 04:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6165(8163243) WEXITSTATUS 1 Nov 05 04:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6165(8163243) done Nov 05 04:56:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8164922 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 #CPUs=288 Partition=day Nov 05 04:56:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6247(8163488) WEXITSTATUS 1 Nov 05 04:56:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6247(8163488) done Nov 05 04:57:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6499(8164423) WEXITSTATUS 1 Nov 05 04:57:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6499(8164423) done Nov 05 04:57:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6769(8164918) WEXITSTATUS 1 Nov 05 04:57:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6769(8164918) done Nov 05 04:57:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3799(8164923) in gpu on r808u14n01 Nov 05 04:57:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3800(8164924) in gpu on r4513u29n01 Nov 05 04:57:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6539(8164489) WEXITSTATUS 1 Nov 05 04:57:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6539(8164489) done Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6773(8164925) in scavenge on r801u30n02 Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6774(8164926) in scavenge on r908u39n03 Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6775(8164927) in scavenge on r912u05n04 Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6776(8164928) in scavenge on r805u25n01 Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6777(8164929) in scavenge on r807u08n01 Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6778(8164930) in scavenge on r906u41n01 Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6779(8164931) in scavenge on r913u09n01 Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6780(8164932) in scavenge on r909u04n04 Nov 05 04:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6781(8164933) in scavenge on r909u10n03 Nov 05 04:57:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6086(8163119) WEXITSTATUS 1 Nov 05 04:57:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6086(8163119) done Nov 05 04:57:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6534(8164479) WEXITSTATUS 1 Nov 05 04:57:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6534(8164479) done Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 04:58:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 04:58:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6371(8163999) WEXITSTATUS 1 Nov 05 04:58:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6371(8163999) done Nov 05 04:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6452(8164306) WEXITSTATUS 1 Nov 05 04:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6452(8164306) done Nov 05 04:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907379 WEXITSTATUS 0 Nov 05 04:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907379 done Nov 05 04:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6328(8163792) WEXITSTATUS 1 Nov 05 04:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6328(8163792) done Nov 05 04:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6580(8164620) WEXITSTATUS 1 Nov 05 04:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6580(8164620) done Nov 05 04:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1758(8157827) WEXITSTATUS 0 Nov 05 04:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1758(8157827) done Nov 05 04:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6512(8164445) WEXITSTATUS 1 Nov 05 04:59:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6512(8164445) done Nov 05 04:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144117 WEXITSTATUS 0 Nov 05 04:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144117 done Nov 05 04:59:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6076(8163103) WEXITSTATUS 1 Nov 05 04:59:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6076(8163103) done Nov 05 04:59:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6717(8164828) WEXITSTATUS 1 Nov 05 04:59:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6717(8164828) done Nov 05 04:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162672 WEXITSTATUS 0 Nov 05 04:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162672 done Nov 05 04:59:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2209(8164934) NodeList=r909u01n03 #CPUs=1 Partition=day Nov 05 04:59:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2210(8164935) NodeList=r913u13n03 #CPUs=1 Partition=day Nov 05 04:59:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144135 NodeList=r801u09n01,r903u04n02 #CPUs=84 Partition=scavenge Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:00:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 05:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 05:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 05:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 05:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 05:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6290(8163643) WEXITSTATUS 1 Nov 05 05:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6290(8163643) done Nov 05 05:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 05 05:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n03 Nov 05 05:00:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907380 in gpu on r805u38n01 Nov 05 05:00:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6782(8164936) in scavenge on r908u33n01 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6783(8164937) in scavenge on r907u01n04 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6784(8164938) in scavenge on r907u05n03 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6785(8164939) in scavenge on r911u05n02 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6786(8164940) in scavenge on r816u15n10 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6787(8164941) in scavenge on r816u15n12 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6788(8164942) in scavenge on r805u25n03 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6789(8164943) in scavenge on r909u13n04 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6790(8164944) in scavenge on r913u09n03 Nov 05 05:00:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6791(8164945) in scavenge on r904u10n03 Nov 05 05:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1712(8156974) WEXITSTATUS 0 Nov 05 05:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1712(8156974) done Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128268 WEXITSTATUS 0 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128268 done Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u06n03 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u11n04 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u09n01 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u11n03 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u06n04 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u11n01 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u11n02 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u09n04 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u09n02 Nov 05 05:00:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u09n03 Nov 05 05:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 05:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 05:00:27 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 05:00:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_5(8158754) WEXITSTATUS 0 Nov 05 05:00:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_5(8158754) done Nov 05 05:00:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6595(8164637) WEXITSTATUS 1 Nov 05 05:00:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6595(8164637) done Nov 05 05:00:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2211(8164946) NodeList=r816u09n05 #CPUs=1 Partition=day Nov 05 05:00:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3800(8164924) WEXITSTATUS 0 Nov 05 05:00:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3800(8164924) done Nov 05 05:01:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6386(8164097) WEXITSTATUS 1 Nov 05 05:01:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6386(8164097) done Nov 05 05:01:13 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2422(8152466) WEXITSTATUS 0 Nov 05 05:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2422(8152466) done Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128271 WEXITSTATUS 0 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128271 done Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u23n04 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u14n01 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u14n02 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u14n03 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u23n02 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u23n01 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u23n03 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u14n04 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u25n01 Nov 05 05:01:17 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u25n02 Nov 05 05:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5139(8158486) WEXITSTATUS 1 Nov 05 05:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5139(8158486) done Nov 05 05:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6656(8164735) WEXITSTATUS 1 Nov 05 05:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6656(8164735) done Nov 05 05:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5078(8158390) WEXITSTATUS 1 Nov 05 05:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5078(8158390) done Nov 05 05:01:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6551(8164504) WEXITSTATUS 1 Nov 05 05:01:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6551(8164504) done Nov 05 05:01:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6662(8164748) WEXITSTATUS 1 Nov 05 05:01:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6662(8164748) done Nov 05 05:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6203(8163339) WEXITSTATUS 1 Nov 05 05:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6203(8163339) done Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:02:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:02:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158969 Nov 05 05:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1524(8144685) WEXITSTATUS 0 Nov 05 05:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1524(8144685) done Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128273 WEXITSTATUS 0 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128273 done Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6545(8164498) WEXITSTATUS 1 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6545(8164498) done Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u27n02 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u27n03 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u30n03 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u30n01 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u25n03 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u25n04 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u27n01 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u30n02 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u27n04 Nov 05 05:02:18 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u30n04 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128274 WEXITSTATUS 0 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128274 done Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u04n01 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n01 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u04n02 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u02n02 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u04n03 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n02 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u04n04 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u02n01 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u02n03 Nov 05 05:02:19 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u02n04 Nov 05 05:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 05:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 05:02:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 05:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6786(8164940) WEXITSTATUS 1 Nov 05 05:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6786(8164940) done Nov 05 05:02:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3801(8164947) in gpu on r4513u29n01 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6792(8164948) in scavenge on r903u01n01 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6793(8164949) in scavenge on r911u24n03 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6794(8164950) in scavenge on r816u15n10 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6795(8164951) in scavenge on r816u29n07 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6796(8164952) in scavenge on r807u05n04 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6797(8164953) in scavenge on r904u10n01 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6798(8164954) in scavenge on r805u27n02 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6799(8164955) in scavenge on r805u27n03 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6800(8164956) in scavenge on r912u32n02 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6801(8164957) in scavenge on r912u32n02 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_723(7572892) in scavenge on r912u32n03 Nov 05 05:02:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_888(7573057) in scavenge on r918u36n03 Nov 05 05:02:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2212(8164958) in scavenge on r906u32n01 Nov 05 05:02:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1024(8128513) WEXITSTATUS 0 Nov 05 05:02:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1024(8128513) done Nov 05 05:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6715(8164826) WEXITSTATUS 1 Nov 05 05:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6715(8164826) done Nov 05 05:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1781(8158038) WEXITSTATUS 0 Nov 05 05:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1781(8158038) done Nov 05 05:03:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3801(8164947) WEXITSTATUS 0 Nov 05 05:03:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3801(8164947) done Nov 05 05:03:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_8(8158169) WEXITSTATUS 1 Nov 05 05:03:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_8(8158169) done Nov 05 05:03:57 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u09n04.grace.ycrc.yale.edu" Nov 05 05:03:57 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u09n04.grace.ycrc.yale.edu" Nov 05 05:03:59 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u09n01.grace.ycrc.yale.edu" Nov 05 05:03:59 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u09n01.grace.ycrc.yale.edu" Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:04:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:04:01 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u06n04.grace.ycrc.yale.edu" Nov 05 05:04:01 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u06n04.grace.ycrc.yale.edu" Nov 05 05:04:01 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u09n02.grace.ycrc.yale.edu" Nov 05 05:04:01 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u09n02.grace.ycrc.yale.edu" Nov 05 05:04:01 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u11n02.grace.ycrc.yale.edu" Nov 05 05:04:01 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u11n02.grace.ycrc.yale.edu" Nov 05 05:04:02 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u09n03.grace.ycrc.yale.edu" Nov 05 05:04:02 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u09n03.grace.ycrc.yale.edu" Nov 05 05:04:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u06n03.grace.ycrc.yale.edu" Nov 05 05:04:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u06n03.grace.ycrc.yale.edu" Nov 05 05:04:05 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u11n01.grace.ycrc.yale.edu" Nov 05 05:04:05 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u11n01.grace.ycrc.yale.edu" Nov 05 05:04:05 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u11n04.grace.ycrc.yale.edu" Nov 05 05:04:05 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u11n04.grace.ycrc.yale.edu" Nov 05 05:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1706(8156851) WEXITSTATUS 0 Nov 05 05:04:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1706(8156851) done Nov 05 05:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6480(8164387) WEXITSTATUS 1 Nov 05 05:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6480(8164387) done Nov 05 05:04:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162676 WEXITSTATUS 0 Nov 05 05:04:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162676 done Nov 05 05:04:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2213(8164959) NodeList=r919u05n02 #CPUs=1 Partition=day Nov 05 05:04:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2214(8164960) NodeList=r815u03n05 #CPUs=1 Partition=day Nov 05 05:04:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2215(8164961) NodeList=r815u03n06 #CPUs=1 Partition=day Nov 05 05:04:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8125755 Nov 05 05:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6431(8164261) WEXITSTATUS 1 Nov 05 05:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6431(8164261) done Nov 05 05:04:51 r912u23n01 slurmctld[163760]: slurmctld: node r803u11n03 returned to service Nov 05 05:04:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6455(8164330) WEXITSTATUS 1 Nov 05 05:04:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6455(8164330) done Nov 05 05:05:02 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u25n01.grace.ycrc.yale.edu" Nov 05 05:05:02 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u25n01.grace.ycrc.yale.edu" Nov 05 05:05:02 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u23n04.grace.ycrc.yale.edu" Nov 05 05:05:02 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u23n04.grace.ycrc.yale.edu" Nov 05 05:05:02 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u25n02.grace.ycrc.yale.edu" Nov 05 05:05:02 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u25n02.grace.ycrc.yale.edu" Nov 05 05:05:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6798(8164954) WEXITSTATUS 1 Nov 05 05:05:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6798(8164954) done Nov 05 05:05:03 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u14n04.grace.ycrc.yale.edu" Nov 05 05:05:03 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u14n04.grace.ycrc.yale.edu" Nov 05 05:05:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u23n01.grace.ycrc.yale.edu" Nov 05 05:05:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u23n01.grace.ycrc.yale.edu" Nov 05 05:05:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u14n01.grace.ycrc.yale.edu" Nov 05 05:05:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u14n01.grace.ycrc.yale.edu" Nov 05 05:05:05 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u23n03.grace.ycrc.yale.edu" Nov 05 05:05:05 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u23n03.grace.ycrc.yale.edu" Nov 05 05:05:05 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u14n03.grace.ycrc.yale.edu" Nov 05 05:05:05 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u14n03.grace.ycrc.yale.edu" Nov 05 05:05:06 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u14n02.grace.ycrc.yale.edu" Nov 05 05:05:06 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u14n02.grace.ycrc.yale.edu" Nov 05 05:05:06 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u23n02.grace.ycrc.yale.edu" Nov 05 05:05:06 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u23n02.grace.ycrc.yale.edu" Nov 05 05:05:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1780(8158037) WEXITSTATUS 0 Nov 05 05:05:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1780(8158037) done Nov 05 05:05:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5765(8162701) WEXITSTATUS 1 Nov 05 05:05:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5765(8162701) done Nov 05 05:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3799(8164923) WEXITSTATUS 0 Nov 05 05:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3799(8164923) done Nov 05 05:05:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123759 WEXITSTATUS 0 Nov 05 05:05:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123759 done Nov 05 05:05:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123763 NodeList=r911u09n03 #CPUs=16 Partition=pi_balou Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3802(8164962) in gpu on r808u14n01 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3803(8164963) in gpu on r4513u29n01 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2216(8164964) in day on r815u15n01 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2217(8164965) in day on r815u15n01 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2218(8164966) in day on r815u15n01 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2219(8164967) in day on r815u15n01 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2220(8164968) in day on r816u15n08 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2221(8164969) in day on r816u15n08 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2222(8164970) in day on r816u15n08 Nov 05 05:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2223(8164971) in day on r816u15n08 Nov 05 05:05:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5678(8162249) WEXITSTATUS 1 Nov 05 05:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5678(8162249) done Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6802(8164972) in scavenge on r907u01n04 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6803(8164973) in scavenge on r801u09n03 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6804(8164974) in scavenge on r903u01n01 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6805(8164975) in scavenge on r903u04n03 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6806(8164976) in scavenge on r805u27n02 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6807(8164977) in scavenge on r914u01n01 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6808(8164978) in scavenge on r918u05n01 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_873(7570535) in scavenge on r816u15n08 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_169(7572337) in scavenge on r816u15n08 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_253(8114519) in scavenge on r815u03n05 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_974(7573143) in scavenge on r815u03n05 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_66(8114332) in scavenge on r815u15n01 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_337(8114604) in scavenge on r815u15n01 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5748(8162609) WEXITSTATUS 1 Nov 05 05:05:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5748(8162609) done Nov 05 05:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6758(8164889) WEXITSTATUS 1 Nov 05 05:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6758(8164889) done Nov 05 05:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144198 WEXITSTATUS 0 Nov 05 05:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144198 done Nov 05 05:05:42 r912u23n01 slurmctld[163760]: slurmctld: node r803u09n04 returned to service Nov 05 05:05:44 r912u23n01 slurmctld[163760]: slurmctld: node r803u09n01 returned to service Nov 05 05:05:46 r912u23n01 slurmctld[163760]: slurmctld: node r803u09n02 returned to service Nov 05 05:05:46 r912u23n01 slurmctld[163760]: slurmctld: node r803u06n04 returned to service Nov 05 05:05:46 r912u23n01 slurmctld[163760]: slurmctld: node r803u11n02 returned to service Nov 05 05:05:48 r912u23n01 slurmctld[163760]: slurmctld: node r803u09n03 returned to service Nov 05 05:05:50 r912u23n01 slurmctld[163760]: slurmctld: node r803u06n03 returned to service Nov 05 05:05:50 r912u23n01 slurmctld[163760]: slurmctld: node r803u11n04 returned to service Nov 05 05:05:50 r912u23n01 slurmctld[163760]: slurmctld: node r803u11n01 returned to service Nov 05 05:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1773(8157957) WEXITSTATUS 0 Nov 05 05:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1773(8157957) done Nov 05 05:06:00 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u30n01.grace.ycrc.yale.edu" Nov 05 05:06:00 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u30n01.grace.ycrc.yale.edu" Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:06:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:06:02 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u30n02.grace.ycrc.yale.edu" Nov 05 05:06:02 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u30n02.grace.ycrc.yale.edu" Nov 05 05:06:02 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u04n03.grace.ycrc.yale.edu" Nov 05 05:06:02 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u04n03.grace.ycrc.yale.edu" Nov 05 05:06:03 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u27n03.grace.ycrc.yale.edu" Nov 05 05:06:03 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u27n03.grace.ycrc.yale.edu" Nov 05 05:06:03 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u25n03.grace.ycrc.yale.edu" Nov 05 05:06:03 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u25n03.grace.ycrc.yale.edu" Nov 05 05:06:03 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u27n01.grace.ycrc.yale.edu" Nov 05 05:06:03 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u27n01.grace.ycrc.yale.edu" Nov 05 05:06:03 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u25n04.grace.ycrc.yale.edu" Nov 05 05:06:03 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u25n04.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1750(8157779) WEXITSTATUS 0 Nov 05 05:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1750(8157779) done Nov 05 05:06:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u02n04.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u02n04.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u04n04.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u04n04.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n01.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n01.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u30n03.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u30n03.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u27n04.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u27n04.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u30n04.grace.ycrc.yale.edu" Nov 05 05:06:04 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u30n04.grace.ycrc.yale.edu" Nov 05 05:06:05 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u02n02.grace.ycrc.yale.edu" Nov 05 05:06:05 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u02n02.grace.ycrc.yale.edu" Nov 05 05:06:05 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n02.grace.ycrc.yale.edu" Nov 05 05:06:05 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n02.grace.ycrc.yale.edu" Nov 05 05:06:06 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u04n01.grace.ycrc.yale.edu" Nov 05 05:06:06 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u04n01.grace.ycrc.yale.edu" Nov 05 05:06:06 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u04n02.grace.ycrc.yale.edu" Nov 05 05:06:06 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u04n02.grace.ycrc.yale.edu" Nov 05 05:06:06 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u27n02.grace.ycrc.yale.edu" Nov 05 05:06:06 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u27n02.grace.ycrc.yale.edu" Nov 05 05:06:07 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u02n01.grace.ycrc.yale.edu" Nov 05 05:06:07 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u02n01.grace.ycrc.yale.edu" Nov 05 05:06:07 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u02n03.grace.ycrc.yale.edu" Nov 05 05:06:07 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u02n03.grace.ycrc.yale.edu" Nov 05 05:06:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5394(8159332) WEXITSTATUS 1 Nov 05 05:06:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5394(8159332) done Nov 05 05:06:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2224(8164979) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 05:06:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2225(8164980) NodeList=r914u05n03 #CPUs=1 Partition=day Nov 05 05:06:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144139 NodeList=r807u01n04,r816u09n11 #CPUs=84 Partition=scavenge Nov 05 05:06:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:06:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6570(8164607) WEXITSTATUS 1 Nov 05 05:06:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6570(8164607) done Nov 05 05:06:47 r912u23n01 slurmctld[163760]: slurmctld: node r803u23n04 returned to service Nov 05 05:06:47 r912u23n01 slurmctld[163760]: slurmctld: node r803u25n02 returned to service Nov 05 05:06:47 r912u23n01 slurmctld[163760]: slurmctld: node r803u25n01 returned to service Nov 05 05:06:48 r912u23n01 slurmctld[163760]: slurmctld: node r803u14n04 returned to service Nov 05 05:06:49 r912u23n01 slurmctld[163760]: slurmctld: node r803u23n01 returned to service Nov 05 05:06:49 r912u23n01 slurmctld[163760]: slurmctld: node r803u14n01 returned to service Nov 05 05:06:50 r912u23n01 slurmctld[163760]: slurmctld: node r803u23n03 returned to service Nov 05 05:06:50 r912u23n01 slurmctld[163760]: slurmctld: node r803u14n03 returned to service Nov 05 05:06:51 r912u23n01 slurmctld[163760]: slurmctld: node r803u23n02 returned to service Nov 05 05:06:51 r912u23n01 slurmctld[163760]: slurmctld: node r803u14n02 returned to service Nov 05 05:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5444(8159659) WEXITSTATUS 1 Nov 05 05:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5444(8159659) done Nov 05 05:07:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6077(8163104) WEXITSTATUS 1 Nov 05 05:07:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6077(8163104) done Nov 05 05:07:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820321 WEXITSTATUS 0 Nov 05 05:07:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820321 done Nov 05 05:07:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820353 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 05:07:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6636(8164711) WEXITSTATUS 1 Nov 05 05:07:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6636(8164711) done Nov 05 05:07:45 r912u23n01 slurmctld[163760]: slurmctld: node r803u30n01 returned to service Nov 05 05:07:47 r912u23n01 slurmctld[163760]: slurmctld: node r803u30n02 returned to service Nov 05 05:07:47 r912u23n01 slurmctld[163760]: slurmctld: node r804u04n03 returned to service Nov 05 05:07:47 r912u23n01 slurmctld[163760]: slurmctld: node r803u27n03 returned to service Nov 05 05:07:48 r912u23n01 slurmctld[163760]: slurmctld: node r803u27n01 returned to service Nov 05 05:07:48 r912u23n01 slurmctld[163760]: slurmctld: node r803u25n04 returned to service Nov 05 05:07:48 r912u23n01 slurmctld[163760]: slurmctld: node r803u25n03 returned to service Nov 05 05:07:49 r912u23n01 slurmctld[163760]: slurmctld: node r804u02n04 returned to service Nov 05 05:07:49 r912u23n01 slurmctld[163760]: slurmctld: node r803u30n04 returned to service Nov 05 05:07:49 r912u23n01 slurmctld[163760]: slurmctld: node r803u27n04 returned to service Nov 05 05:07:49 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n01 returned to service Nov 05 05:07:49 r912u23n01 slurmctld[163760]: slurmctld: node r804u04n04 returned to service Nov 05 05:07:49 r912u23n01 slurmctld[163760]: slurmctld: node r803u30n03 returned to service Nov 05 05:07:50 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n02 returned to service Nov 05 05:07:50 r912u23n01 slurmctld[163760]: slurmctld: node r804u02n02 returned to service Nov 05 05:07:51 r912u23n01 slurmctld[163760]: slurmctld: node r803u27n02 returned to service Nov 05 05:07:51 r912u23n01 slurmctld[163760]: slurmctld: node r804u04n01 returned to service Nov 05 05:07:51 r912u23n01 slurmctld[163760]: slurmctld: node r804u04n02 returned to service Nov 05 05:07:52 r912u23n01 slurmctld[163760]: slurmctld: node r804u02n03 returned to service Nov 05 05:07:52 r912u23n01 slurmctld[163760]: slurmctld: node r804u02n01 returned to service Nov 05 05:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820328 WEXITSTATUS 0 Nov 05 05:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820328 done Nov 05 05:07:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820354 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6780(8164932) WEXITSTATUS 1 Nov 05 05:08:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6780(8164932) done Nov 05 05:08:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1797(8158155) WEXITSTATUS 0 Nov 05 05:08:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1797(8158155) done Nov 05 05:08:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6809(8164981) in scavenge on r801u06n01 Nov 05 05:08:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6810(8164982) in scavenge on r802u32n04 Nov 05 05:08:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6811(8164983) in scavenge on r816u29n11 Nov 05 05:08:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6812(8164984) in scavenge on r904u04n01 Nov 05 05:08:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6813(8164985) in scavenge on r909u04n04 Nov 05 05:08:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6814(8164986) in scavenge on r909u29n03 Nov 05 05:08:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6815(8164987) in scavenge on r918u09n01 Nov 05 05:08:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6816(8164988) in scavenge on r919u01n04 Nov 05 05:08:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2226(8164989) in scavenge on r906u39n01 Nov 05 05:08:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2542(8152809) WEXITSTATUS 0 Nov 05 05:08:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2542(8152809) done Nov 05 05:08:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1658(8156243) WEXITSTATUS 0 Nov 05 05:08:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1658(8156243) done Nov 05 05:08:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162646 WEXITSTATUS 0 Nov 05 05:08:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162646 done Nov 05 05:08:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_20(8158124) WEXITSTATUS 1 Nov 05 05:08:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_20(8158124) done Nov 05 05:08:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6437(8164267) WEXITSTATUS 1 Nov 05 05:08:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6437(8164267) done Nov 05 05:08:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2227(8164990) NodeList=r815u35n01 #CPUs=1 Partition=day Nov 05 05:08:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2228(8164991) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 05:08:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2229(8164992) NodeList=r815u15n05 #CPUs=1 Partition=day Nov 05 05:08:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6749(8164880) WEXITSTATUS 1 Nov 05 05:08:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6749(8164880) done Nov 05 05:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6528(8164473) WEXITSTATUS 1 Nov 05 05:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6528(8164473) done Nov 05 05:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3748(8164324) WEXITSTATUS 0 Nov 05 05:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3748(8164324) done Nov 05 05:09:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5694(8162316) WEXITSTATUS 1 Nov 05 05:09:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5694(8162316) done Nov 05 05:09:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1670(8156303) WEXITSTATUS 0 Nov 05 05:09:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1670(8156303) done Nov 05 05:09:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2230(8164993) NodeList=r816u15n04 #CPUs=1 Partition=day Nov 05 05:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_14(8163292) WEXITSTATUS 0 Nov 05 05:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_14(8163292) done Nov 05 05:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6626(8164691) WEXITSTATUS 1 Nov 05 05:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6626(8164691) done Nov 05 05:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6382(8164093) WEXITSTATUS 1 Nov 05 05:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6382(8164093) done Nov 05 05:09:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820327 WEXITSTATUS 0 Nov 05 05:09:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820327 done Nov 05 05:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820355 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 05:09:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_16(8163294) WEXITSTATUS 0 Nov 05 05:09:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_16(8163294) done Nov 05 05:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1675(8156308) WEXITSTATUS 0 Nov 05 05:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1675(8156308) done Nov 05 05:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820322 WEXITSTATUS 0 Nov 05 05:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820322 done Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:10:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:10:01 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820356 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 05:10:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6496(8164420) WEXITSTATUS 1 Nov 05 05:10:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6496(8164420) done Nov 05 05:10:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5623(8161724) WEXITSTATUS 1 Nov 05 05:10:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5623(8161724) done Nov 05 05:10:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2420(8152464) WEXITSTATUS 0 Nov 05 05:10:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2420(8152464) done Nov 05 05:10:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_15(8163293) WEXITSTATUS 0 Nov 05 05:10:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_15(8163293) done Nov 05 05:10:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2231(8164994) NodeList=r815u15n04 #CPUs=1 Partition=day Nov 05 05:10:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6347(8163850) WEXITSTATUS 1 Nov 05 05:10:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6347(8163850) done Nov 05 05:10:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3804(8164995) in gpu on r808u08n01 Nov 05 05:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3803(8164963) WEXITSTATUS 0 Nov 05 05:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3803(8164963) done Nov 05 05:10:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6817(8164996) in scavenge on r801u06n03 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6818(8164997) in scavenge on r802u02n02 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6819(8164998) in scavenge on r816u29n07 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6820(8164999) in scavenge on r816u09n10 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6821(8165000) in scavenge on r816u15n11 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6822(8165001) in scavenge on r807u05n01 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6823(8165002) in scavenge on r909u13n03 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6824(8165003) in scavenge on r919u01n04 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6825(8165004) in scavenge on r919u05n03 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_381(8114648) in scavenge on r815u15n05 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_727(8114996) in scavenge on r815u23n06 Nov 05 05:10:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1130(7573299) in scavenge on r816u15n04 Nov 05 05:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2232(8165005) in scavenge on r815u03n11 Nov 05 05:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2233(8165006) in scavenge on r904u13n01 Nov 05 05:11:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5061(8158366) WEXITSTATUS 1 Nov 05 05:11:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5061(8158366) done Nov 05 05:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_17(8163295) WEXITSTATUS 0 Nov 05 05:11:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_17(8163295) done Nov 05 05:11:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6514(8164449) WEXITSTATUS 1 Nov 05 05:11:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6514(8164449) done Nov 05 05:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_290(8163776) WEXITSTATUS 0 Nov 05 05:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_290(8163776) done Nov 05 05:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1674(8156307) WEXITSTATUS 0 Nov 05 05:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1674(8156307) done Nov 05 05:11:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2234(8165007) NodeList=r815u09n08 #CPUs=1 Partition=day Nov 05 05:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820325 WEXITSTATUS 0 Nov 05 05:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820325 done Nov 05 05:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820357 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:12:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:12:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1652(8156186) WEXITSTATUS 0 Nov 05 05:12:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1652(8156186) done Nov 05 05:12:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6785(8164939) WEXITSTATUS 1 Nov 05 05:12:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6785(8164939) done Nov 05 05:12:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:12:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2235(8165008) NodeList=r918u32n04 #CPUs=1 Partition=day Nov 05 05:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5843(8162814) WEXITSTATUS 1 Nov 05 05:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5843(8162814) done Nov 05 05:12:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8126147 Nov 05 05:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6542(8164492) WEXITSTATUS 1 Nov 05 05:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6542(8164492) done Nov 05 05:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6498(8164422) WEXITSTATUS 1 Nov 05 05:12:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6498(8164422) done Nov 05 05:12:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6184(8163283) WEXITSTATUS 1 Nov 05 05:12:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6184(8163283) done Nov 05 05:12:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1679(8156312) WEXITSTATUS 0 Nov 05 05:12:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1679(8156312) done Nov 05 05:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_4(8158247) WEXITSTATUS 1 Nov 05 05:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_4(8158247) done Nov 05 05:13:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5717(8162482) WEXITSTATUS 1 Nov 05 05:13:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5717(8162482) done Nov 05 05:13:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3805(8165009) in gpu on r4513u29n01 Nov 05 05:13:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2236(8165010) in day on r914u01n03 Nov 05 05:13:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2237(8165011) in day on r815u23n06 Nov 05 05:13:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2238(8165012) in day on r815u29n06 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6826(8165013) in scavenge on r908u33n02 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6827(8165014) in scavenge on r801u27n02 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6828(8165015) in scavenge on r802u11n02 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6829(8165016) in scavenge on r911u05n02 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6830(8165017) in scavenge on r912u36n01 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6831(8165018) in scavenge on r914u01n02 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6832(8165019) in scavenge on r914u01n03 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6833(8165020) in scavenge on r914u05n02 Nov 05 05:13:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_613(8116086) in scavenge on r815u15n07 Nov 05 05:13:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1654(8156219) WEXITSTATUS 0 Nov 05 05:13:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1654(8156219) done Nov 05 05:13:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 05:13:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2239(8165021) NodeList=r815u35n04 #CPUs=1 Partition=day Nov 05 05:13:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1289(8135279) WEXITSTATUS 0 Nov 05 05:13:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1289(8135279) done Nov 05 05:13:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2413(8152457) WEXITSTATUS 0 Nov 05 05:13:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2413(8152457) done Nov 05 05:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6828(8165015) WEXITSTATUS 1 Nov 05 05:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6828(8165015) done Nov 05 05:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1708(8156853) WEXITSTATUS 0 Nov 05 05:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1708(8156853) done Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:14:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:14:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162634 WEXITSTATUS 0 Nov 05 05:14:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162634 done Nov 05 05:14:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_953(8128294) WEXITSTATUS 0 Nov 05 05:14:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_953(8128294) done Nov 05 05:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6294(8163647) WEXITSTATUS 1 Nov 05 05:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6294(8163647) done Nov 05 05:14:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3604(8160124) WEXITSTATUS 0 Nov 05 05:14:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3604(8160124) done Nov 05 05:14:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165022 InitPrio=31893 usec=3817 Nov 05 05:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6376(8164047) WEXITSTATUS 1 Nov 05 05:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6376(8164047) done Nov 05 05:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4819(8157785) WEXITSTATUS 1 Nov 05 05:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4819(8157785) done Nov 05 05:14:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2240(8165023) NodeList=r918u01n04 #CPUs=1 Partition=day Nov 05 05:14:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2241(8165024) NodeList=r918u36n03 #CPUs=1 Partition=day Nov 05 05:14:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2242(8165025) NodeList=r815u35n06 #CPUs=1 Partition=day Nov 05 05:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162655 WEXITSTATUS 0 Nov 05 05:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162655 done Nov 05 05:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_13(8163291) WEXITSTATUS 0 Nov 05 05:15:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_13(8163291) done Nov 05 05:15:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6284(8163603) WEXITSTATUS 1 Nov 05 05:15:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6284(8163603) done Nov 05 05:15:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5474(8159897) WEXITSTATUS 1 Nov 05 05:15:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5474(8159897) done Nov 05 05:15:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_24(8163324) WEXITSTATUS 0 Nov 05 05:15:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_24(8163324) done Nov 05 05:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6287(8163606) WEXITSTATUS 1 Nov 05 05:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6287(8163606) done Nov 05 05:15:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2243(8165026) NodeList=r815u03n08 #CPUs=1 Partition=day Nov 05 05:15:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2244(8165027) NodeList=r815u03n08 #CPUs=1 Partition=day Nov 05 05:15:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6787(8164941) WEXITSTATUS 1 Nov 05 05:15:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6787(8164941) done Nov 05 05:15:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_22(8163322) WEXITSTATUS 0 Nov 05 05:15:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_22(8163322) done Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:16:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:16:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3806(8165028) in gpu on r4513u20n01 Nov 05 05:16:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6834(8165029) in scavenge on r802u11n02 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6835(8165030) in scavenge on r815u23n07 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6836(8165031) in scavenge on r816u15n12 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6837(8165032) in scavenge on r816u23n12 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6838(8165033) in scavenge on r907u28n01 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6839(8165034) in scavenge on r805u25n04 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6840(8165035) in scavenge on r918u05n01 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6841(8165036) in scavenge on r918u32n04 Nov 05 05:16:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_122(7572289) in scavenge on r815u03n08 Nov 05 05:16:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2245(8165037) in scavenge on r815u03n12 Nov 05 05:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6802(8164972) WEXITSTATUS 1 Nov 05 05:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6802(8164972) done Nov 05 05:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6841(8165036) WEXITSTATUS 1 Nov 05 05:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6841(8165036) done Nov 05 05:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6820(8164999) WEXITSTATUS 1 Nov 05 05:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6820(8164999) done Nov 05 05:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_20(8163298) WEXITSTATUS 0 Nov 05 05:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_20(8163298) done Nov 05 05:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6836(8165031) WEXITSTATUS 1 Nov 05 05:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6836(8165031) done Nov 05 05:17:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_46(8163380) WEXITSTATUS 0 Nov 05 05:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_46(8163380) done Nov 05 05:17:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2456(8127483) Nov 05 05:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162675 WEXITSTATUS 0 Nov 05 05:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162675 done Nov 05 05:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1739(8157742) WEXITSTATUS 0 Nov 05 05:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1739(8157742) done Nov 05 05:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_6(8163247) WEXITSTATUS 0 Nov 05 05:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_6(8163247) done Nov 05 05:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5533(8160868) WEXITSTATUS 1 Nov 05 05:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5533(8160868) done Nov 05 05:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6721(8164832) WEXITSTATUS 1 Nov 05 05:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6721(8164832) done Nov 05 05:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_50(8163384) WEXITSTATUS 0 Nov 05 05:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_50(8163384) done Nov 05 05:17:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1800(8158158) WEXITSTATUS 0 Nov 05 05:17:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1800(8158158) done Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:18:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3789(8164821) WEXITSTATUS 0 Nov 05 05:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3789(8164821) done Nov 05 05:18:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820324 WEXITSTATUS 0 Nov 05 05:18:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820324 done Nov 05 05:18:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820358 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 05:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3606(8160326) WEXITSTATUS 0 Nov 05 05:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3606(8160326) done Nov 05 05:18:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6823(8165002) WEXITSTATUS 1 Nov 05 05:18:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6823(8165002) done Nov 05 05:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4284(8154050) WEXITSTATUS 1 Nov 05 05:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4284(8154050) done Nov 05 05:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6809(8164981) WEXITSTATUS 1 Nov 05 05:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6809(8164981) done Nov 05 05:18:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2246(8165038) NodeList=r904u07n03 #CPUs=1 Partition=day Nov 05 05:18:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2247(8165039) NodeList=r909u29n03 #CPUs=1 Partition=day Nov 05 05:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3807(8165040) in gpu on r808u14n01 Nov 05 05:18:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3808(8165041) in gpu on r808u14n01 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6842(8165042) in scavenge on r907u01n04 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6843(8165043) in scavenge on r802u23n02 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6844(8165044) in scavenge on r801u06n01 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6845(8165045) in scavenge on r801u14n01 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6846(8165046) in scavenge on r815u03n09 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6847(8165047) in scavenge on r816u09n10 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6848(8165048) in scavenge on r816u15n12 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6849(8165049) in scavenge on r805u14n04 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6850(8165050) in scavenge on r909u13n03 Nov 05 05:18:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6851(8165051) in scavenge on r918u32n04 Nov 05 05:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3807(8165040) WEXITSTATUS 0 Nov 05 05:18:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3807(8165040) done Nov 05 05:18:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2248(8165052) in scavenge on r801u14n01 Nov 05 05:18:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2249(8165053) in scavenge on r815u03n11 Nov 05 05:18:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2250(8165054) in scavenge on r904u13n01 Nov 05 05:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1798(8158156) WEXITSTATUS 0 Nov 05 05:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1798(8158156) done Nov 05 05:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6628(8164693) WEXITSTATUS 1 Nov 05 05:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6628(8164693) done Nov 05 05:19:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_5(8163225) WEXITSTATUS 0 Nov 05 05:19:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_5(8163225) done Nov 05 05:19:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144133 WEXITSTATUS 0 Nov 05 05:19:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144133 done Nov 05 05:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6522(8164460) WEXITSTATUS 1 Nov 05 05:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6522(8164460) done Nov 05 05:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_27(8163327) WEXITSTATUS 0 Nov 05 05:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_27(8163327) done Nov 05 05:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1672(8156305) WEXITSTATUS 0 Nov 05 05:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1672(8156305) done Nov 05 05:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_47(8163381) WEXITSTATUS 0 Nov 05 05:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_47(8163381) done Nov 05 05:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_19(8158180) WEXITSTATUS 1 Nov 05 05:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_19(8158180) done Nov 05 05:19:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2251(8165055) NodeList=r815u09n01 #CPUs=1 Partition=day Nov 05 05:19:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2252(8165056) NodeList=r815u15n04 #CPUs=1 Partition=day Nov 05 05:19:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144138 NodeList=r816u09n12,r917u01n01 #CPUs=96 Partition=scavenge Nov 05 05:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_44(8163378) WEXITSTATUS 0 Nov 05 05:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_44(8163378) done Nov 05 05:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_21(8163299) WEXITSTATUS 0 Nov 05 05:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_21(8163299) done Nov 05 05:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1847(8158684) WEXITSTATUS 0 Nov 05 05:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1847(8158684) done Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:20:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_872(8127986) WEXITSTATUS 0 Nov 05 05:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_872(8127986) done Nov 05 05:20:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_11(8163289) WEXITSTATUS 0 Nov 05 05:20:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_11(8163289) done Nov 05 05:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_9(8163270) WEXITSTATUS 0 Nov 05 05:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_9(8163270) done Nov 05 05:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3808(8165041) WEXITSTATUS 0 Nov 05 05:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3808(8165041) done Nov 05 05:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6273(8163555) WEXITSTATUS 1 Nov 05 05:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6273(8163555) done Nov 05 05:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_23(8163323) WEXITSTATUS 0 Nov 05 05:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_23(8163323) done Nov 05 05:20:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1678(8156311) WEXITSTATUS 0 Nov 05 05:20:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1678(8156311) done Nov 05 05:20:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2253(8165057) NodeList=r815u23n05 #CPUs=1 Partition=day Nov 05 05:20:38 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 05:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3265(8133508) WEXITSTATUS 1 Nov 05 05:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3265(8133508) done Nov 05 05:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6435(8164265) WEXITSTATUS 1 Nov 05 05:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6435(8164265) done Nov 05 05:21:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_13(8158174) WEXITSTATUS 1 Nov 05 05:21:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_13(8158174) done Nov 05 05:21:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1799(8158157) WEXITSTATUS 0 Nov 05 05:21:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1799(8158157) done Nov 05 05:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_8(8163269) WEXITSTATUS 0 Nov 05 05:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_8(8163269) done Nov 05 05:21:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6806(8164976) WEXITSTATUS 1 Nov 05 05:21:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6806(8164976) done Nov 05 05:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3809(8165058) in gpu on r808u14n01 Nov 05 05:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3810(8165059) in gpu on r808u14n01 Nov 05 05:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2254(8165060) in day on r815u03n06 Nov 05 05:21:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:21:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_25(8163325) WEXITSTATUS 0 Nov 05 05:21:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_25(8163325) done Nov 05 05:21:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6852(8165061) in scavenge on r802u06n03 Nov 05 05:21:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6853(8165062) in scavenge on r816u35n11 Nov 05 05:21:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6854(8165063) in scavenge on r805u27n02 Nov 05 05:21:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6855(8165064) in scavenge on r806u09n04 Nov 05 05:21:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6856(8165065) in scavenge on r807u03n03 Nov 05 05:21:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6857(8165066) in scavenge on r919u05n04 Nov 05 05:21:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_68(8114334) in scavenge on r815u03n06 Nov 05 05:21:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3810(8165059) WEXITSTATUS 0 Nov 05 05:21:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3810(8165059) done Nov 05 05:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_43(8163377) WEXITSTATUS 0 Nov 05 05:21:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_43(8163377) done Nov 05 05:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5529(8160864) WEXITSTATUS 1 Nov 05 05:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5529(8160864) done Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:22:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5954(8162950) WEXITSTATUS 1 Nov 05 05:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5954(8162950) done Nov 05 05:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_30(8163330) WEXITSTATUS 0 Nov 05 05:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_30(8163330) done Nov 05 05:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_19(8163297) WEXITSTATUS 0 Nov 05 05:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_19(8163297) done Nov 05 05:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6676(8164762) WEXITSTATUS 1 Nov 05 05:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6676(8164762) done Nov 05 05:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1661(8156294) WEXITSTATUS 0 Nov 05 05:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1661(8156294) done Nov 05 05:22:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1650(8156106) WEXITSTATUS 0 Nov 05 05:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1650(8156106) done Nov 05 05:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6560(8164582) WEXITSTATUS 1 Nov 05 05:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6560(8164582) done Nov 05 05:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_52(8163386) WEXITSTATUS 0 Nov 05 05:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_52(8163386) done Nov 05 05:22:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_28(8163328) WEXITSTATUS 0 Nov 05 05:22:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_28(8163328) done Nov 05 05:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_18(8163296) WEXITSTATUS 0 Nov 05 05:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_18(8163296) done Nov 05 05:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6497(8164421) WEXITSTATUS 1 Nov 05 05:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6497(8164421) done Nov 05 05:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1721(8157215) WEXITSTATUS 0 Nov 05 05:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1721(8157215) done Nov 05 05:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_59(8135758) WEXITSTATUS 0 Nov 05 05:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_59(8135758) done Nov 05 05:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1796(8158154) WEXITSTATUS 0 Nov 05 05:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1796(8158154) done Nov 05 05:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_82(8163456) WEXITSTATUS 0 Nov 05 05:23:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_82(8163456) done Nov 05 05:23:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2255(8165067) NodeList=r805u27n04 #CPUs=1 Partition=day Nov 05 05:23:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2256(8165068) NodeList=r919u32n04 #CPUs=1 Partition=day Nov 05 05:23:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2257(8165069) NodeList=r816u03n03 #CPUs=1 Partition=day Nov 05 05:23:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6815(8164987) WEXITSTATUS 1 Nov 05 05:23:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6815(8164987) done Nov 05 05:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_12(8163290) WEXITSTATUS 0 Nov 05 05:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_12(8163290) done Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:24:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:24:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3811(8165070) in gpu on r808u14n01 Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6450(8164304) WEXITSTATUS 1 Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6450(8164304) done Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6858(8165071) in scavenge on r801u23n03 Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6859(8165072) in scavenge on r816u15n10 Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6860(8165073) in scavenge on r815u03n11 Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6861(8165074) in scavenge on r815u03n12 Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6862(8165075) in scavenge on r909u07n01 Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6863(8165076) in scavenge on r909u13n04 Nov 05 05:24:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6864(8165077) in scavenge on r918u05n03 Nov 05 05:24:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6848(8165048) WEXITSTATUS 1 Nov 05 05:24:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6848(8165048) done Nov 05 05:24:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2258(8165078) in scavenge on r904u13n01 Nov 05 05:24:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2259(8165079) in scavenge on r904u13n01 Nov 05 05:24:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2260(8165080) in scavenge on r904u13n01 Nov 05 05:24:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 05:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1644(8155918) WEXITSTATUS 0 Nov 05 05:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1644(8155918) done Nov 05 05:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6665(8164751) WEXITSTATUS 1 Nov 05 05:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6665(8164751) done Nov 05 05:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_45(8163379) WEXITSTATUS 0 Nov 05 05:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_45(8163379) done Nov 05 05:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_54(8163388) WEXITSTATUS 0 Nov 05 05:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_54(8163388) done Nov 05 05:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1714(8157093) WEXITSTATUS 0 Nov 05 05:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1714(8157093) done Nov 05 05:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3771(8164631) WEXITSTATUS 0 Nov 05 05:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3771(8164631) done Nov 05 05:24:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_81(8163455) WEXITSTATUS 0 Nov 05 05:24:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_81(8163455) done Nov 05 05:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_31(8163331) WEXITSTATUS 0 Nov 05 05:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_31(8163331) done Nov 05 05:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2261(8165081) NodeList=r815u15n07 #CPUs=1 Partition=day Nov 05 05:24:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2262(8165082) NodeList=r816u09n03 #CPUs=1 Partition=day Nov 05 05:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1795(8158106) WEXITSTATUS 0 Nov 05 05:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1795(8158106) done Nov 05 05:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_83(8163457) WEXITSTATUS 0 Nov 05 05:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_83(8163457) done Nov 05 05:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_60(8163420) WEXITSTATUS 0 Nov 05 05:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_60(8163420) done Nov 05 05:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820326 WEXITSTATUS 0 Nov 05 05:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820326 done Nov 05 05:25:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820359 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 05:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1529(8145435) WEXITSTATUS 0 Nov 05 05:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1529(8145435) done Nov 05 05:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6846(8165046) WEXITSTATUS 1 Nov 05 05:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6846(8165046) done Nov 05 05:25:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES mem grp_used_tres_run_secs underflow, tried to remove 3932160 seconds when only 2613248 remained. Nov 05 05:25:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 32 remained. Nov 05 05:25:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 32 remained. Nov 05 05:25:18 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES mem grp_used_tres_run_secs underflow, tried to remove 3932160 seconds when only 2613248 remained. Nov 05 05:25:18 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 32 remained. Nov 05 05:25:18 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 32 remained. Nov 05 05:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1655(8156220) WEXITSTATUS 0 Nov 05 05:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1655(8156220) done Nov 05 05:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1698(8156720) WEXITSTATUS 0 Nov 05 05:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1698(8156720) done Nov 05 05:25:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5027(8158323) WEXITSTATUS 1 Nov 05 05:25:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5027(8158323) done Nov 05 05:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6510(8164443) WEXITSTATUS 1 Nov 05 05:25:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6510(8164443) done Nov 05 05:25:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6182(8163281) WEXITSTATUS 1 Nov 05 05:25:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6182(8163281) done Nov 05 05:25:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2263(8165083) NodeList=r815u09n01 #CPUs=1 Partition=day Nov 05 05:25:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2264(8165084) NodeList=r816u03n03 #CPUs=1 Partition=day Nov 05 05:25:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2265(8165085) NodeList=r816u09n08 #CPUs=1 Partition=day Nov 05 05:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6818(8164997) WEXITSTATUS 1 Nov 05 05:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6818(8164997) done Nov 05 05:25:43 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8139112 Nov 05 05:25:43 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES cpu grp_used_tres_run_secs underflow, tried to remove 64 seconds when only 28 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES mem grp_used_tres_run_secs underflow, tried to remove 524288 seconds when only 0 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES node grp_used_tres_run_secs underflow, tried to remove 8 seconds when only 0 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES billing grp_used_tres_run_secs underflow, tried to remove 800 seconds when only 260 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 8 seconds when only 0 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES cpu grp_used_tres_run_secs underflow, tried to remove 64 seconds when only 28 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES mem grp_used_tres_run_secs underflow, tried to remove 524288 seconds when only 0 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES node grp_used_tres_run_secs underflow, tried to remove 8 seconds when only 0 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES billing grp_used_tres_run_secs underflow, tried to remove 800 seconds when only 260 remained. Nov 05 05:25:43 r912u23n01 slurmctld[163760]: error: _handle_qos_tres_run_secs: job 8139112: QOS part_gpu_devel TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 8 seconds when only 0 remained. Nov 05 05:25:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_26(8163326) WEXITSTATUS 0 Nov 05 05:25:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_26(8163326) done Nov 05 05:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5138(8158485) WEXITSTATUS 1 Nov 05 05:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5138(8158485) done Nov 05 05:25:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6753(8164884) WEXITSTATUS 1 Nov 05 05:25:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6753(8164884) done Nov 05 05:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820329 WEXITSTATUS 0 Nov 05 05:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820329 done Nov 05 05:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820360 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 05:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_10(8163288) WEXITSTATUS 0 Nov 05 05:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_10(8163288) done Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:26:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:26:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4833(8157812) WEXITSTATUS 1 Nov 05 05:26:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4833(8157812) done Nov 05 05:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1775(8157970) WEXITSTATUS 0 Nov 05 05:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1775(8157970) done Nov 05 05:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_118(8163510) WEXITSTATUS 0 Nov 05 05:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_118(8163510) done Nov 05 05:26:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2266(8165086) NodeList=r816u03n01 #CPUs=1 Partition=day Nov 05 05:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_113(8163505) WEXITSTATUS 0 Nov 05 05:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_113(8163505) done Nov 05 05:26:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3812(8165087) in gpu on r4513u23n01 Nov 05 05:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6865(8165088) in scavenge on r908u39n04 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6866(8165089) in scavenge on r802u02n02 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6867(8165090) in scavenge on r915u35n02 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6868(8165091) in scavenge on r904u01n02 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6869(8165092) in scavenge on r904u01n03 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6870(8165093) in scavenge on r816u15n12 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6871(8165094) in scavenge on r816u35n12 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6872(8165095) in scavenge on r815u03n09 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6873(8165096) in scavenge on r815u03n11 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6874(8165097) in scavenge on r906u39n01 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_981(7571947) in scavenge on r815u35n07 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1056(7572022) in scavenge on r805u27n04 Nov 05 05:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_604(7571569) in scavenge on r919u32n01 Nov 05 05:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6801(8164957) WEXITSTATUS 1 Nov 05 05:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6801(8164957) done Nov 05 05:26:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2267(8165098) in scavenge on r815u23n07 Nov 05 05:26:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2268(8165099) in scavenge on r815u03n11 Nov 05 05:26:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2269(8165100) in scavenge on r815u03n12 Nov 05 05:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6729(8164844) WEXITSTATUS 1 Nov 05 05:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6729(8164844) done Nov 05 05:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_4(8162994) WEXITSTATUS 0 Nov 05 05:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_4(8162994) done Nov 05 05:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_96(8163470) WEXITSTATUS 0 Nov 05 05:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_96(8163470) done Nov 05 05:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_49(8163383) WEXITSTATUS 0 Nov 05 05:27:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_49(8163383) done Nov 05 05:27:15 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158575 Nov 05 05:27:15 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8159563 Nov 05 05:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_35(8163351) WEXITSTATUS 0 Nov 05 05:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_35(8163351) done Nov 05 05:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6844(8165044) WEXITSTATUS 1 Nov 05 05:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6844(8165044) done Nov 05 05:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6340(8163843) WEXITSTATUS 1 Nov 05 05:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6340(8163843) done Nov 05 05:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6356(8163900) WEXITSTATUS 1 Nov 05 05:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6356(8163900) done Nov 05 05:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6320(8163750) WEXITSTATUS 1 Nov 05 05:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6320(8163750) done Nov 05 05:27:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:27:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1144(8132432) WEXITSTATUS 0 Nov 05 05:27:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1144(8132432) done Nov 05 05:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1788(8158051) WEXITSTATUS 0 Nov 05 05:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1788(8158051) done Nov 05 05:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6659(8164738) WEXITSTATUS 1 Nov 05 05:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6659(8164738) done Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_34(8163350) WEXITSTATUS 0 Nov 05 05:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_34(8163350) done Nov 05 05:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6833(8165020) WEXITSTATUS 1 Nov 05 05:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6833(8165020) done Nov 05 05:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_116(8163508) WEXITSTATUS 0 Nov 05 05:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_116(8163508) done Nov 05 05:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_37(8163353) WEXITSTATUS 0 Nov 05 05:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_37(8163353) done Nov 05 05:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1737(8157740) WEXITSTATUS 0 Nov 05 05:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1737(8157740) done Nov 05 05:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_7(8163268) WEXITSTATUS 0 Nov 05 05:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_7(8163268) done Nov 05 05:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_95(8163469) WEXITSTATUS 0 Nov 05 05:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_95(8163469) done Nov 05 05:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2495(8127554) WEXITSTATUS 1 Nov 05 05:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2495(8127554) done Nov 05 05:28:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2270(8165101) NodeList=r806u14n03 #CPUs=1 Partition=day Nov 05 05:28:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2271(8165102) NodeList=r815u23n01 #CPUs=1 Partition=day Nov 05 05:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5770(8162706) WEXITSTATUS 1 Nov 05 05:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5770(8162706) done Nov 05 05:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_53(8163387) WEXITSTATUS 0 Nov 05 05:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_53(8163387) done Nov 05 05:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_955(8128296) WEXITSTATUS 0 Nov 05 05:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_955(8128296) done Nov 05 05:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3811(8165070) WEXITSTATUS 0 Nov 05 05:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3811(8165070) done Nov 05 05:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_41(8163357) WEXITSTATUS 0 Nov 05 05:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_41(8163357) done Nov 05 05:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_91(8163465) WEXITSTATUS 0 Nov 05 05:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_91(8163465) done Nov 05 05:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1663(8156296) WEXITSTATUS 0 Nov 05 05:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1663(8156296) done Nov 05 05:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6687(8164789) WEXITSTATUS 1 Nov 05 05:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6687(8164789) done Nov 05 05:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_32(8163348) WEXITSTATUS 0 Nov 05 05:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_32(8163348) done Nov 05 05:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820333 WEXITSTATUS 0 Nov 05 05:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820333 done Nov 05 05:29:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_61(8163421) WEXITSTATUS 0 Nov 05 05:29:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_61(8163421) done Nov 05 05:29:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820361 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 05:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6690(8164792) WEXITSTATUS 1 Nov 05 05:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6690(8164792) done Nov 05 05:29:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:29:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:29:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:29:20 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:29:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:29:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:29:23 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:29:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3813(8165103) in gpu on r808u14n01 Nov 05 05:29:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2272(8165104) in day on r913u13n01 Nov 05 05:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_51(8163385) WEXITSTATUS 0 Nov 05 05:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_51(8163385) done Nov 05 05:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_112(8163504) WEXITSTATUS 0 Nov 05 05:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_112(8163504) done Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6875(8165105) in scavenge on r801u06n01 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6876(8165106) in scavenge on r802u06n04 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6877(8165107) in scavenge on r916u04n02 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6878(8165108) in scavenge on r815u09n11 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6879(8165109) in scavenge on r816u29n11 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6880(8165110) in scavenge on r816u35n10 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6881(8165111) in scavenge on r909u07n04 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6882(8165112) in scavenge on r914u05n02 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6883(8165113) in scavenge on r912u32n02 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6884(8165114) in scavenge on r913u05n03 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6885(8165115) in scavenge on r918u09n01 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6886(8165116) in scavenge on r918u32n04 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_234(8114500) in scavenge on r907u24n01 Nov 05 05:29:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_261(8114527) in scavenge on r904u13n01 Nov 05 05:29:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_1(8158161) WEXITSTATUS 1 Nov 05 05:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_1(8158161) done Nov 05 05:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2428(8152472) WEXITSTATUS 0 Nov 05 05:29:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2428(8152472) done Nov 05 05:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1660(8156293) WEXITSTATUS 0 Nov 05 05:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1660(8156293) done Nov 05 05:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1699(8156721) WEXITSTATUS 0 Nov 05 05:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1699(8156721) done Nov 05 05:29:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1785(8158048) WEXITSTATUS 0 Nov 05 05:29:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1785(8158048) done Nov 05 05:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1701(8156846) WEXITSTATUS 0 Nov 05 05:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1701(8156846) done Nov 05 05:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162648 WEXITSTATUS 0 Nov 05 05:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162648 done Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:30:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6859(8165072) WEXITSTATUS 1 Nov 05 05:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6859(8165072) done Nov 05 05:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_36(8163352) WEXITSTATUS 0 Nov 05 05:30:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_36(8163352) done Nov 05 05:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820331 WEXITSTATUS 0 Nov 05 05:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820331 done Nov 05 05:30:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820362 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 05:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1821(8158459) WEXITSTATUS 0 Nov 05 05:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1821(8158459) done Nov 05 05:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1519(8143999) WEXITSTATUS 0 Nov 05 05:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1519(8143999) done Nov 05 05:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820323 WEXITSTATUS 0 Nov 05 05:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820323 done Nov 05 05:30:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820363 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 05:30:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 05:30:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 05:30:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2273(8165117) NodeList=r805u04n02 #CPUs=1 Partition=day Nov 05 05:30:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2274(8165118) NodeList=r807u03n01 #CPUs=1 Partition=day Nov 05 05:30:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2275(8165119) NodeList=r909u29n01 #CPUs=1 Partition=day Nov 05 05:30:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2276(8165120) NodeList=r904u10n03 #CPUs=1 Partition=day Nov 05 05:30:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2277(8165121) NodeList=r918u05n03 #CPUs=1 Partition=day Nov 05 05:30:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2278(8165122) NodeList=r919u05n01 #CPUs=1 Partition=day Nov 05 05:30:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2279(8165123) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 05:30:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6470(8164375) WEXITSTATUS 1 Nov 05 05:30:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6470(8164375) done Nov 05 05:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6686(8164788) WEXITSTATUS 1 Nov 05 05:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6686(8164788) done Nov 05 05:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 05:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 05:30:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 05:30:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6170(8163256) WEXITSTATUS 1 Nov 05 05:30:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6170(8163256) done Nov 05 05:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154644 WEXITSTATUS 0 Nov 05 05:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154644 done Nov 05 05:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_29(8163329) WEXITSTATUS 0 Nov 05 05:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_29(8163329) done Nov 05 05:31:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_149(8163565) WEXITSTATUS 0 Nov 05 05:31:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_149(8163565) done Nov 05 05:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_40(8163356) WEXITSTATUS 0 Nov 05 05:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_40(8163356) done Nov 05 05:31:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6204(8163340) WEXITSTATUS 1 Nov 05 05:31:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6204(8163340) done Nov 05 05:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1665(8156298) WEXITSTATUS 0 Nov 05 05:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1665(8156298) done Nov 05 05:31:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154714 NodeList=r816u03n02 #CPUs=16 Partition=day Nov 05 05:31:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2280(8165124) NodeList=r919u09n02 #CPUs=1 Partition=day Nov 05 05:31:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_873(8127987) WEXITSTATUS 0 Nov 05 05:31:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_873(8127987) done Nov 05 05:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1740(8157743) WEXITSTATUS 0 Nov 05 05:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1740(8157743) done Nov 05 05:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_93(8163467) WEXITSTATUS 0 Nov 05 05:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_93(8163467) done Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:32:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:32:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2281(8165125) in day on r904u04n01 Nov 05 05:32:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1790(8158053) WEXITSTATUS 0 Nov 05 05:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1790(8158053) done Nov 05 05:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6887(8165126) in scavenge on r908u39n01 Nov 05 05:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6888(8165127) in scavenge on r801u14n01 Nov 05 05:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6889(8165128) in scavenge on r816u15n10 Nov 05 05:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6890(8165129) in scavenge on r815u03n11 Nov 05 05:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6891(8165130) in scavenge on r815u03n12 Nov 05 05:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_673(8114942) in scavenge on r912u28n03 Nov 05 05:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_281(8114547) in scavenge on r815u35n06 Nov 05 05:32:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6804(8164974) WEXITSTATUS 1 Nov 05 05:32:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6804(8164974) done Nov 05 05:32:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4898(8158031) WEXITSTATUS 1 Nov 05 05:32:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4898(8158031) done Nov 05 05:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1764(8157870) WEXITSTATUS 0 Nov 05 05:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1764(8157870) done Nov 05 05:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6681(8164779) WEXITSTATUS 1 Nov 05 05:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6681(8164779) done Nov 05 05:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6872(8165095) WEXITSTATUS 1 Nov 05 05:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6872(8165095) done Nov 05 05:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6253(8163494) WEXITSTATUS 1 Nov 05 05:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6253(8163494) done Nov 05 05:32:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2282(8165131) NodeList=r805u04n01 #CPUs=1 Partition=day Nov 05 05:32:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2283(8165132) NodeList=r815u09n08 #CPUs=1 Partition=day Nov 05 05:32:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_67(8163427) WEXITSTATUS 0 Nov 05 05:32:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_67(8163427) done Nov 05 05:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6747(8164872) WEXITSTATUS 1 Nov 05 05:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6747(8164872) done Nov 05 05:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_76(8163436) WEXITSTATUS 0 Nov 05 05:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_76(8163436) done Nov 05 05:32:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_48(8163382) WEXITSTATUS 0 Nov 05 05:32:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_48(8163382) done Nov 05 05:32:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1765(8157871) WEXITSTATUS 0 Nov 05 05:32:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1765(8157871) done Nov 05 05:32:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3813(8165103) WEXITSTATUS 0 Nov 05 05:32:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3813(8165103) done Nov 05 05:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_119(8163511) WEXITSTATUS 0 Nov 05 05:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_119(8163511) done Nov 05 05:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_117(8163509) WEXITSTATUS 0 Nov 05 05:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_117(8163509) done Nov 05 05:33:14 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2480(8127532) Nov 05 05:33:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6397(8164138) WEXITSTATUS 1 Nov 05 05:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6397(8164138) done Nov 05 05:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5231(8158634) WEXITSTATUS 1 Nov 05 05:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5231(8158634) done Nov 05 05:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6885(8165115) WEXITSTATUS 1 Nov 05 05:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6885(8165115) done Nov 05 05:33:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_64(8163424) WEXITSTATUS 0 Nov 05 05:33:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_64(8163424) done Nov 05 05:33:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6261(8163543) WEXITSTATUS 1 Nov 05 05:33:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6261(8163543) done Nov 05 05:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 05:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 05:33:27 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 05:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_33(8163349) WEXITSTATUS 0 Nov 05 05:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_33(8163349) done Nov 05 05:33:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2284(8165133) NodeList=r806u09n04 #CPUs=1 Partition=day Nov 05 05:33:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144131 NodeList=r801u32n03,r807u35n01 #CPUs=68 Partition=scavenge Nov 05 05:33:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1820(8158458) WEXITSTATUS 0 Nov 05 05:33:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1820(8158458) done Nov 05 05:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6867(8165090) WEXITSTATUS 1 Nov 05 05:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6867(8165090) done Nov 05 05:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6767(8164916) WEXITSTATUS 1 Nov 05 05:33:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6767(8164916) done Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_68(8163428) WEXITSTATUS 0 Nov 05 05:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_68(8163428) done Nov 05 05:34:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1787(8158050) WEXITSTATUS 0 Nov 05 05:34:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1787(8158050) done Nov 05 05:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162691 WEXITSTATUS 0 Nov 05 05:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162691 done Nov 05 05:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1715(8157094) WEXITSTATUS 0 Nov 05 05:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1715(8157094) done Nov 05 05:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2285(8165134) NodeList=r909u04n03 #CPUs=1 Partition=day Nov 05 05:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2286(8165135) NodeList=r904u10n01 #CPUs=1 Partition=day Nov 05 05:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2287(8165136) NodeList=r919u09n04 #CPUs=1 Partition=day Nov 05 05:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2288(8165137) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 05:34:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3814(8165138) in gpu on r808u14n01 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6892(8165139) in scavenge on r802u02n01 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6893(8165140) in scavenge on r903u01n01 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6894(8165141) in scavenge on r915u35n02 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6895(8165142) in scavenge on r815u03n09 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6896(8165143) in scavenge on r816u29n07 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6897(8165144) in scavenge on r816u35n10 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6898(8165145) in scavenge on r903u34n01 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6899(8165146) in scavenge on r807u03n01 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6900(8165147) in scavenge on r807u11n01 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6901(8165148) in scavenge on r805u02n01 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6902(8165149) in scavenge on r805u09n03 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6903(8165150) in scavenge on r805u25n02 Nov 05 05:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6904(8165151) in scavenge on r918u09n01 Nov 05 05:34:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2289(8165152) in scavenge on r815u03n12 Nov 05 05:34:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2290(8165153) in scavenge on r816u35n04 Nov 05 05:34:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2291(8165154) in scavenge on r904u13n01 Nov 05 05:34:54 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 05:34:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6707(8164812) WEXITSTATUS 1 Nov 05 05:34:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6707(8164812) done Nov 05 05:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_217(8163669) WEXITSTATUS 0 Nov 05 05:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_217(8163669) done Nov 05 05:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6714(8164825) WEXITSTATUS 1 Nov 05 05:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6714(8164825) done Nov 05 05:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_69(8163429) WEXITSTATUS 0 Nov 05 05:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_69(8163429) done Nov 05 05:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1528(8145318) WEXITSTATUS 0 Nov 05 05:34:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1528(8145318) done Nov 05 05:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1859(8160126) WEXITSTATUS 0 Nov 05 05:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1859(8160126) done Nov 05 05:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1724(8157448) WEXITSTATUS 0 Nov 05 05:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1724(8157448) done Nov 05 05:35:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6557(8164579) WEXITSTATUS 1 Nov 05 05:35:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6557(8164579) done Nov 05 05:35:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_77(8163437) WEXITSTATUS 0 Nov 05 05:35:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_77(8163437) done Nov 05 05:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6901(8165148) WEXITSTATUS 1 Nov 05 05:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6901(8165148) done Nov 05 05:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2292(8165155) NodeList=r816u03n01 #CPUs=1 Partition=day Nov 05 05:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2293(8165156) NodeList=r816u03n02 #CPUs=1 Partition=day Nov 05 05:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_65(8163425) WEXITSTATUS 0 Nov 05 05:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_65(8163425) done Nov 05 05:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6164(8163242) WEXITSTATUS 1 Nov 05 05:35:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6164(8163242) done Nov 05 05:35:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3804(8164995) WEXITSTATUS 0 Nov 05 05:35:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3804(8164995) done Nov 05 05:35:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8159782 Nov 05 05:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_66(8163426) WEXITSTATUS 0 Nov 05 05:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_66(8163426) done Nov 05 05:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_62(8163422) WEXITSTATUS 0 Nov 05 05:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_62(8163422) done Nov 05 05:35:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5494(8160331) WEXITSTATUS 1 Nov 05 05:35:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5494(8160331) done Nov 05 05:35:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6635(8164700) WEXITSTATUS 1 Nov 05 05:35:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6635(8164700) done Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154647 WEXITSTATUS 0 Nov 05 05:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154647 done Nov 05 05:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1770(8157938) WEXITSTATUS 0 Nov 05 05:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1770(8157938) done Nov 05 05:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_84(8163458) WEXITSTATUS 0 Nov 05 05:36:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_84(8163458) done Nov 05 05:36:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_63(8163423) WEXITSTATUS 0 Nov 05 05:36:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_63(8163423) done Nov 05 05:36:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5382(8159275) WEXITSTATUS 1 Nov 05 05:36:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5382(8159275) done Nov 05 05:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_39(8163355) WEXITSTATUS 0 Nov 05 05:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_39(8163355) done Nov 05 05:36:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_86(8163460) WEXITSTATUS 0 Nov 05 05:36:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_86(8163460) done Nov 05 05:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6154(8163201) WEXITSTATUS 1 Nov 05 05:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6154(8163201) done Nov 05 05:36:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154715 NodeList=r909u01n02 #CPUs=16 Partition=day Nov 05 05:36:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2294(8165157) NodeList=r909u10n02 #CPUs=1 Partition=day Nov 05 05:36:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_153(8163569) WEXITSTATUS 0 Nov 05 05:36:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_153(8163569) done Nov 05 05:36:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_114(8163506) WEXITSTATUS 0 Nov 05 05:36:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_114(8163506) done Nov 05 05:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1710(8156972) WEXITSTATUS 0 Nov 05 05:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1710(8156972) done Nov 05 05:37:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6874(8165097) WEXITSTATUS 1 Nov 05 05:37:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6874(8165097) done Nov 05 05:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5642(8161844) WEXITSTATUS 1 Nov 05 05:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5642(8161844) done Nov 05 05:37:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3815(8165158) in gpu on r808u08n01 Nov 05 05:37:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2295(8165159) in day on r919u09n04 Nov 05 05:37:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_70(8163430) WEXITSTATUS 0 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_70(8163430) done Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6905(8165160) in scavenge on r908u39n02 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6906(8165161) in scavenge on r802u11n01 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6907(8165162) in scavenge on r802u11n03 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6908(8165163) in scavenge on r815u03n11 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6909(8165164) in scavenge on r815u23n09 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6910(8165165) in scavenge on r906u39n01 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6911(8165166) in scavenge on r806u09n02 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6912(8165167) in scavenge on r909u04n04 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6913(8165168) in scavenge on r805u02n01 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6914(8165169) in scavenge on r805u14n01 Nov 05 05:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6915(8165170) in scavenge on r919u05n01 Nov 05 05:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_85(8163459) WEXITSTATUS 0 Nov 05 05:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_85(8163459) done Nov 05 05:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_71(8163431) WEXITSTATUS 0 Nov 05 05:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_71(8163431) done Nov 05 05:37:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 05:37:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1684(8156458) WEXITSTATUS 0 Nov 05 05:37:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1684(8156458) done Nov 05 05:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6723(8164838) WEXITSTATUS 1 Nov 05 05:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6723(8164838) done Nov 05 05:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6122(8163163) WEXITSTATUS 1 Nov 05 05:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6122(8163163) done Nov 05 05:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_89(8163463) WEXITSTATUS 0 Nov 05 05:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_89(8163463) done Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:38:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6908(8165163) WEXITSTATUS 1 Nov 05 05:38:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6908(8165163) done Nov 05 05:38:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_373(7572541) Nov 05 05:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1713(8157092) WEXITSTATUS 0 Nov 05 05:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1713(8157092) done Nov 05 05:38:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_954(8128295) WEXITSTATUS 0 Nov 05 05:38:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_954(8128295) done Nov 05 05:38:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_152(8163568) WEXITSTATUS 0 Nov 05 05:38:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_152(8163568) done Nov 05 05:38:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3815(8165158) WEXITSTATUS 0 Nov 05 05:38:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3815(8165158) done Nov 05 05:38:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2296(8165171) NodeList=r909u10n04 #CPUs=1 Partition=day Nov 05 05:38:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2297(8165172) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 05:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_115(8163507) WEXITSTATUS 0 Nov 05 05:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_115(8163507) done Nov 05 05:38:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6706(8164811) WEXITSTATUS 1 Nov 05 05:38:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6706(8164811) done Nov 05 05:38:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:38:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165173 InitPrio=20041 usec=3847 Nov 05 05:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164922 WEXITSTATUS 0 Nov 05 05:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164922 done Nov 05 05:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5788(8162743) WEXITSTATUS 1 Nov 05 05:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5788(8162743) done Nov 05 05:39:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6614(8164674) WEXITSTATUS 1 Nov 05 05:39:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6614(8164674) done Nov 05 05:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5188(8158551) WEXITSTATUS 1 Nov 05 05:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5188(8158551) done Nov 05 05:39:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6094(8163127) WEXITSTATUS 1 Nov 05 05:39:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6094(8163127) done Nov 05 05:39:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165173 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 #CPUs=288 Partition=day Nov 05 05:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3814(8165138) WEXITSTATUS 0 Nov 05 05:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3814(8165138) done Nov 05 05:39:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_151(8163567) WEXITSTATUS 0 Nov 05 05:39:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_151(8163567) done Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3816(8165174) in gpu on r808u08n01 Nov 05 05:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3817(8165175) in gpu on r808u14n01 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1784(8158047) WEXITSTATUS 0 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1784(8158047) done Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6916(8165176) in scavenge on r906u01n02 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6917(8165177) in scavenge on r815u03n11 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6918(8165178) in scavenge on r815u03n12 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6919(8165179) in scavenge on r815u29n09 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6920(8165180) in scavenge on r815u29n12 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6921(8165181) in scavenge on r902u07n01 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6922(8165182) in scavenge on r904u04n02 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6923(8165183) in scavenge on r912u36n02 Nov 05 05:40:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_376(8114643) in scavenge on r815u09n12 Nov 05 05:40:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6680(8164778) WEXITSTATUS 1 Nov 05 05:40:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6680(8164778) done Nov 05 05:40:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2298(8165184) in scavenge on r815u03n11 Nov 05 05:40:16 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 05:40:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_139(8163531) WEXITSTATUS 0 Nov 05 05:40:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_139(8163531) done Nov 05 05:40:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3816(8165174) WEXITSTATUS 0 Nov 05 05:40:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3816(8165174) done Nov 05 05:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_55(8163389) WEXITSTATUS 0 Nov 05 05:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_55(8163389) done Nov 05 05:40:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2299(8165185) NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 05:40:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_87(8163461) WEXITSTATUS 0 Nov 05 05:40:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_87(8163461) done Nov 05 05:40:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162668 WEXITSTATUS 0 Nov 05 05:40:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162668 done Nov 05 05:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1673(8156306) WEXITSTATUS 0 Nov 05 05:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1673(8156306) done Nov 05 05:40:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6041(8163063) WEXITSTATUS 1 Nov 05 05:40:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6041(8163063) done Nov 05 05:41:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6415(8164188) WEXITSTATUS 1 Nov 05 05:41:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6415(8164188) done Nov 05 05:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6918(8165178) WEXITSTATUS 1 Nov 05 05:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6918(8165178) done Nov 05 05:41:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1789(8158052) WEXITSTATUS 0 Nov 05 05:41:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1789(8158052) done Nov 05 05:41:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2491(8127550) Nov 05 05:41:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2300(8165186) NodeList=r918u05n02 #CPUs=1 Partition=day Nov 05 05:41:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2301(8165187) NodeList=r815u09n03 #CPUs=1 Partition=day Nov 05 05:41:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2302(8165188) NodeList=r815u09n04 #CPUs=1 Partition=day Nov 05 05:41:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6890(8165129) WEXITSTATUS 1 Nov 05 05:41:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6890(8165129) done Nov 05 05:41:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_78(8163438) WEXITSTATUS 0 Nov 05 05:41:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_78(8163438) done Nov 05 05:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1647(8155984) WEXITSTATUS 0 Nov 05 05:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1647(8155984) done Nov 05 05:41:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_58(8163392) WEXITSTATUS 0 Nov 05 05:41:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_58(8163392) done Nov 05 05:41:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1730(8157680) WEXITSTATUS 0 Nov 05 05:41:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1730(8157680) done Nov 05 05:41:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162809 WEXITSTATUS 0 Nov 05 05:41:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162809 done Nov 05 05:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1805(8158218) WEXITSTATUS 0 Nov 05 05:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1805(8158218) done Nov 05 05:41:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1731(8157681) WEXITSTATUS 0 Nov 05 05:41:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1731(8157681) done Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:42:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1783(8158046) WEXITSTATUS 0 Nov 05 05:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1783(8158046) done Nov 05 05:42:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820334 WEXITSTATUS 0 Nov 05 05:42:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820334 done Nov 05 05:42:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820364 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 05:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1688(8156582) WEXITSTATUS 0 Nov 05 05:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1688(8156582) done Nov 05 05:42:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820332 WEXITSTATUS 0 Nov 05 05:42:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820332 done Nov 05 05:42:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820365 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 05:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_42(8163358) WEXITSTATUS 0 Nov 05 05:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_42(8163358) done Nov 05 05:42:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_38(8163354) WEXITSTATUS 0 Nov 05 05:42:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_38(8163354) done Nov 05 05:42:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2303(8165189) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 05:42:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2304(8165190) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 05:42:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2305(8165191) NodeList=r909u04n03 #CPUs=1 Partition=day Nov 05 05:42:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2306(8165192) NodeList=r909u13n02 #CPUs=1 Partition=day Nov 05 05:42:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2307(8165193) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 05:42:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1669(8156302) WEXITSTATUS 0 Nov 05 05:42:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1669(8156302) done Nov 05 05:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_137(8163529) WEXITSTATUS 0 Nov 05 05:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_137(8163529) done Nov 05 05:42:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3818(8165194) in gpu on r808u08n01 Nov 05 05:42:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2308(8165195) in day on r919u36n01 Nov 05 05:42:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_79(8163453) WEXITSTATUS 0 Nov 05 05:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_79(8163453) done Nov 05 05:42:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6924(8165196) in scavenge on r911u09n01 Nov 05 05:42:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6925(8165197) in scavenge on r815u23n09 Nov 05 05:42:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6926(8165198) in scavenge on r815u03n11 Nov 05 05:42:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6927(8165199) in scavenge on r815u03n12 Nov 05 05:42:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6928(8165200) in scavenge on r912u36n02 Nov 05 05:42:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6929(8165201) in scavenge on r914u01n01 Nov 05 05:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_169(8163585) WEXITSTATUS 0 Nov 05 05:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_169(8163585) done Nov 05 05:42:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 05:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_216(8163668) WEXITSTATUS 0 Nov 05 05:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_216(8163668) done Nov 05 05:42:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1811(8158352) WEXITSTATUS 0 Nov 05 05:42:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1811(8158352) done Nov 05 05:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6892(8165139) WEXITSTATUS 1 Nov 05 05:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6892(8165139) done Nov 05 05:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_71(8135770) WEXITSTATUS 0 Nov 05 05:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_71(8135770) done Nov 05 05:43:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_120(8163512) WEXITSTATUS 0 Nov 05 05:43:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_120(8163512) done Nov 05 05:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6657(8164736) WEXITSTATUS 1 Nov 05 05:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6657(8164736) done Nov 05 05:43:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6600(8164642) WEXITSTATUS 1 Nov 05 05:43:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6600(8164642) done Nov 05 05:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_138(8163530) WEXITSTATUS 0 Nov 05 05:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_138(8163530) done Nov 05 05:43:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6057(8163080) WEXITSTATUS 1 Nov 05 05:43:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6057(8163080) done Nov 05 05:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_148(8163564) WEXITSTATUS 0 Nov 05 05:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_148(8163564) done Nov 05 05:43:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6185(8163284) WEXITSTATUS 1 Nov 05 05:43:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6185(8163284) done Nov 05 05:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_80(8163454) WEXITSTATUS 0 Nov 05 05:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_80(8163454) done Nov 05 05:43:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_150(8163566) WEXITSTATUS 0 Nov 05 05:43:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_150(8163566) done Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:44:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2167(8151959) WEXITSTATUS 0 Nov 05 05:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2167(8151959) done Nov 05 05:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_88(8163462) WEXITSTATUS 0 Nov 05 05:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_88(8163462) done Nov 05 05:44:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6601(8164643) WEXITSTATUS 1 Nov 05 05:44:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6601(8164643) done Nov 05 05:44:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3345(8135094) WEXITSTATUS 1 Nov 05 05:44:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3345(8135094) done Nov 05 05:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1729(8157679) WEXITSTATUS 0 Nov 05 05:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1729(8157679) done Nov 05 05:44:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:44:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2309(8165202) NodeList=r806u14n01 #CPUs=1 Partition=day Nov 05 05:44:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1540(8146143) WEXITSTATUS 0 Nov 05 05:44:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1540(8146143) done Nov 05 05:45:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6854(8165063) WEXITSTATUS 1 Nov 05 05:45:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6854(8165063) done Nov 05 05:45:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6759(8164890) WEXITSTATUS 1 Nov 05 05:45:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6759(8164890) done Nov 05 05:45:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3818(8165194) WEXITSTATUS 0 Nov 05 05:45:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3818(8165194) done Nov 05 05:45:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6402(8164175) WEXITSTATUS 1 Nov 05 05:45:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6402(8164175) done Nov 05 05:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1772(8157940) WEXITSTATUS 0 Nov 05 05:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1772(8157940) done Nov 05 05:45:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6869(8165092) WEXITSTATUS 1 Nov 05 05:45:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6869(8165092) done Nov 05 05:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5728(8162561) WEXITSTATUS 1 Nov 05 05:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5728(8162561) done Nov 05 05:45:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6728(8164843) WEXITSTATUS 1 Nov 05 05:45:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6728(8164843) done Nov 05 05:45:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165203 InitPrio=20705 usec=5535 Nov 05 05:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1865(8160863) WEXITSTATUS 0 Nov 05 05:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1865(8160863) done Nov 05 05:45:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_94(8163468) WEXITSTATUS 0 Nov 05 05:45:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_94(8163468) done Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_0(8165204) in day on r805u11n04 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_1(8165205) in day on r918u36n02 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_2(8165206) in day on r806u09n01 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_3(8165207) in day on r806u11n01 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_4(8165208) in day on r807u05n02 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_5(8165209) in day on r807u05n03 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_6(8165210) in day on r807u08n04 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3819(8165211) in gpu on r808u08n01 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2310(8165212) in day on r816u03n07 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6825(8165004) WEXITSTATUS 1 Nov 05 05:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6825(8165004) done Nov 05 05:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1727(8157677) WEXITSTATUS 0 Nov 05 05:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1727(8157677) done Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6930(8165213) in scavenge on r802u02n01 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6931(8165214) in scavenge on r904u01n03 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6932(8165215) in scavenge on r916u01n01 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6933(8165216) in scavenge on r815u03n12 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6934(8165217) in scavenge on r816u15n12 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6935(8165218) in scavenge on r906u41n02 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6936(8165219) in scavenge on r913u01n02 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6937(8165220) in scavenge on r913u05n01 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6938(8165221) in scavenge on r805u23n04 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6939(8165222) in scavenge on r805u27n02 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6940(8165223) in scavenge on r805u27n03 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6941(8165224) in scavenge on r918u05n01 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6942(8165225) in scavenge on r918u05n02 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6943(8165226) in scavenge on r918u09n04 Nov 05 05:45:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_647(8114916) in scavenge on r918u36n02 Nov 05 05:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_75(8163435) WEXITSTATUS 0 Nov 05 05:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_75(8163435) done Nov 05 05:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1418(8135951) WEXITSTATUS 0 Nov 05 05:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1418(8135951) done Nov 05 05:45:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3071(8130265) WEXITSTATUS 1 Nov 05 05:45:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3071(8130265) done Nov 05 05:45:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6390(8164101) WEXITSTATUS 1 Nov 05 05:45:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6390(8164101) done Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:46:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:46:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1827(8158483) WEXITSTATUS 0 Nov 05 05:46:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1827(8158483) done Nov 05 05:46:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_121(8163513) WEXITSTATUS 0 Nov 05 05:46:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_121(8163513) done Nov 05 05:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1754(8157796) WEXITSTATUS 0 Nov 05 05:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1754(8157796) done Nov 05 05:46:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820330 WEXITSTATUS 0 Nov 05 05:46:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820330 done Nov 05 05:46:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_258(8114524) Nov 05 05:46:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_661(8114930) Nov 05 05:46:17 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6101(8163139) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:46:17 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6102(8163140) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820366 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 05:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1709(8156854) WEXITSTATUS 0 Nov 05 05:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1709(8156854) done Nov 05 05:46:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6102(8163140) Nov 05 05:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2595(8152862) WEXITSTATUS 0 Nov 05 05:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2595(8152862) done Nov 05 05:46:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6101(8163139) Nov 05 05:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_7(8165227) NodeList=r908u37n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:46:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5261(8158695) WEXITSTATUS 1 Nov 05 05:46:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5261(8158695) done Nov 05 05:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907589 WEXITSTATUS 0 Nov 05 05:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907589 done Nov 05 05:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6736(8164857) WEXITSTATUS 1 Nov 05 05:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6736(8164857) done Nov 05 05:46:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_8(8165228) NodeList=r816u09n03 #CPUs=16 Partition=day Nov 05 05:46:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_578(7571543) WEXITSTATUS 0 Nov 05 05:46:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_578(7571543) done Nov 05 05:47:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6920(8165180) WEXITSTATUS 1 Nov 05 05:47:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6920(8165180) done Nov 05 05:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1541(8146144) WEXITSTATUS 0 Nov 05 05:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1541(8146144) done Nov 05 05:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_57(8163391) WEXITSTATUS 0 Nov 05 05:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_57(8163391) done Nov 05 05:47:17 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6115(8163156) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:47:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6115(8163156) Nov 05 05:47:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_9(8165229) NodeList=r908u35n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_956(8128297) WEXITSTATUS 0 Nov 05 05:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_956(8128297) done Nov 05 05:47:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_141(8163533) WEXITSTATUS 0 Nov 05 05:47:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_141(8163533) done Nov 05 05:47:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_10(8165230) NodeList=r816u03n01 #CPUs=16 Partition=day Nov 05 05:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_856(7571822) WEXITSTATUS 0 Nov 05 05:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_856(7571822) done Nov 05 05:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6443(8164273) WEXITSTATUS 1 Nov 05 05:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6443(8164273) done Nov 05 05:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1810(8124675) WEXITSTATUS 1 Nov 05 05:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1810(8124675) done Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1728(8157678) WEXITSTATUS 0 Nov 05 05:48:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1728(8157678) done Nov 05 05:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1844(8158629) WEXITSTATUS 0 Nov 05 05:48:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1844(8158629) done Nov 05 05:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6360(8163951) WEXITSTATUS 1 Nov 05 05:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6360(8163951) done Nov 05 05:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2311(8165231) in day on r805u02n02 Nov 05 05:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2312(8165232) in day on r913u24n04 Nov 05 05:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2313(8165233) in day on r806u04n04 Nov 05 05:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2314(8165234) in day on r806u23n02 Nov 05 05:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2315(8165235) in day on r815u03n07 Nov 05 05:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_841(7573010) WEXITSTATUS 0 Nov 05 05:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_841(7573010) done Nov 05 05:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_631(8114900) WEXITSTATUS 0 Nov 05 05:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_631(8114900) done Nov 05 05:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6944(8165236) in scavenge on r802u02n02 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6945(8165237) in scavenge on r911u24n03 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6946(8165238) in scavenge on r815u29n12 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6947(8165239) in scavenge on r816u23n08 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6948(8165240) in scavenge on r806u09n03 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6949(8165241) in scavenge on r918u01n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6950(8165242) in scavenge on r918u01n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6951(8165243) in scavenge on r918u01n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6952(8165244) in scavenge on r918u05n03 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6953(8165245) in scavenge on r919u05n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6954(8165246) in scavenge on r919u05n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_11(8165247) in scavenge on r801u25n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_12(8165248) in scavenge on r903u01n03 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_13(8165249) in scavenge on r916u04n03 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_14(8165250) in scavenge on r802u25n04 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_15(8165251) in scavenge on r802u27n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_16(8165252) in scavenge on r911u01n03 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_17(8165253) in scavenge on r912u01n04 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_18(8165254) in scavenge on r904u01n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_19(8165255) in scavenge on r904u01n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_20(8165256) in scavenge on r916u04n04 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_21(8165257) in scavenge on r815u03n10 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_22(8165258) in scavenge on r807u01n03 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_23(8165259) in scavenge on r815u35n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_746(8115015) in scavenge on r805u34n01 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_190(8163625) WEXITSTATUS 0 Nov 05 05:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_190(8163625) done Nov 05 05:48:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1554(8147565) WEXITSTATUS 0 Nov 05 05:48:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1554(8147565) done Nov 05 05:48:18 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6118(8163159) has been requeued to reclaim resources for JobId=8165203_24(8165203) Nov 05 05:48:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8160538 Nov 05 05:48:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6118(8163159) Nov 05 05:48:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_24(8165260) NodeList=r908u37n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6905(8165160) WEXITSTATUS 1 Nov 05 05:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6905(8165160) done Nov 05 05:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_1009(8116484) WEXITSTATUS 0 Nov 05 05:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_1009(8116484) done Nov 05 05:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6589(8164629) WEXITSTATUS 1 Nov 05 05:48:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6589(8164629) done Nov 05 05:48:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_25(8165261) NodeList=r816u09n03 #CPUs=16 Partition=day Nov 05 05:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_44(7572211) WEXITSTATUS 0 Nov 05 05:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_44(7572211) done Nov 05 05:48:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_182(8163617) WEXITSTATUS 0 Nov 05 05:48:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_182(8163617) done Nov 05 05:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_294(8163780) WEXITSTATUS 0 Nov 05 05:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_294(8163780) done Nov 05 05:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_74(8163434) WEXITSTATUS 0 Nov 05 05:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_74(8163434) done Nov 05 05:48:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165262 InitPrio=20705 usec=3171 Nov 05 05:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1751(8157780) WEXITSTATUS 0 Nov 05 05:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1751(8157780) done Nov 05 05:48:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6752(8164883) WEXITSTATUS 1 Nov 05 05:48:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6752(8164883) done Nov 05 05:49:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_192(8163627) WEXITSTATUS 0 Nov 05 05:49:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_192(8163627) done Nov 05 05:49:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6292(8163645) WEXITSTATUS 1 Nov 05 05:49:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6292(8163645) done Nov 05 05:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_166(8163582) WEXITSTATUS 0 Nov 05 05:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_166(8163582) done Nov 05 05:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1689(8156583) WEXITSTATUS 0 Nov 05 05:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1689(8156583) done Nov 05 05:49:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_107(8163481) WEXITSTATUS 0 Nov 05 05:49:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_107(8163481) done Nov 05 05:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_90(8163464) WEXITSTATUS 0 Nov 05 05:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_90(8163464) done Nov 05 05:49:18 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6162(8163240) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:49:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6162(8163240) Nov 05 05:49:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:49:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_26(8165263) NodeList=r908u37n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:49:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6364(8163955) WEXITSTATUS 1 Nov 05 05:49:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6364(8163955) done Nov 05 05:49:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_27(8165264) NodeList=r805u02n03 #CPUs=16 Partition=day Nov 05 05:49:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_28(8165265) NodeList=r815u23n02 #CPUs=16 Partition=day Nov 05 05:49:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_106(7572273) WEXITSTATUS 0 Nov 05 05:49:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_106(7572273) done Nov 05 05:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_125(8163517) WEXITSTATUS 0 Nov 05 05:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_125(8163517) done Nov 05 05:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1812(8158353) WEXITSTATUS 0 Nov 05 05:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1812(8158353) done Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:50:01 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:50:02 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:50:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:50:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 05:50:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1725(8157675) WEXITSTATUS 0 Nov 05 05:50:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1725(8157675) done Nov 05 05:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_111(8163503) WEXITSTATUS 0 Nov 05 05:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_111(8163503) done Nov 05 05:50:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_10(8158171) WEXITSTATUS 1 Nov 05 05:50:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_10(8158171) done Nov 05 05:50:18 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6147(8163194) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:50:18 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6327(8163791) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6221(8163373) WEXITSTATUS 1 Nov 05 05:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6221(8163373) done Nov 05 05:50:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6327(8163791) Nov 05 05:50:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6147(8163194) Nov 05 05:50:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_97(8163471) WEXITSTATUS 0 Nov 05 05:50:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_97(8163471) done Nov 05 05:50:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_29(8165266) NodeList=r908u37n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:50:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6458(8164333) WEXITSTATUS 1 Nov 05 05:50:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6458(8164333) done Nov 05 05:50:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144179 WEXITSTATUS 0 Nov 05 05:50:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144179 done Nov 05 05:50:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6724(8164839) WEXITSTATUS 1 Nov 05 05:50:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6724(8164839) done Nov 05 05:50:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1843(8158628) WEXITSTATUS 0 Nov 05 05:50:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1843(8158628) done Nov 05 05:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6527(8164472) WEXITSTATUS 1 Nov 05 05:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6527(8164472) done Nov 05 05:50:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_30(8165267) NodeList=r815u09n01 #CPUs=16 Partition=day Nov 05 05:50:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_882(7571848) WEXITSTATUS 0 Nov 05 05:50:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_882(7571848) done Nov 05 05:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_167(8163583) WEXITSTATUS 0 Nov 05 05:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_167(8163583) done Nov 05 05:50:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_144(8163560) WEXITSTATUS 0 Nov 05 05:50:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_144(8163560) done Nov 05 05:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_109(8163501) WEXITSTATUS 0 Nov 05 05:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_109(8163501) done Nov 05 05:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1825(8158481) WEXITSTATUS 0 Nov 05 05:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1825(8158481) done Nov 05 05:50:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6915(8165170) WEXITSTATUS 1 Nov 05 05:50:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6915(8165170) done Nov 05 05:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1676(8156309) WEXITSTATUS 0 Nov 05 05:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1676(8156309) done Nov 05 05:50:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_31(8165268) in day on r815u03n02 Nov 05 05:50:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165203_32(8165269) in day on r815u23n01 Nov 05 05:50:53 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6433(8164263) has been requeued to reclaim resources for JobId=8165203_33(8165203) Nov 05 05:50:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_873(7570535) WEXITSTATUS 0 Nov 05 05:50:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_873(7570535) done Nov 05 05:50:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_169(7572337) WEXITSTATUS 0 Nov 05 05:50:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_169(7572337) done Nov 05 05:50:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2316(8165270) in day on r805u09n02 Nov 05 05:50:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2317(8165271) in day on r918u01n04 Nov 05 05:50:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2318(8165272) in day on r815u03n02 Nov 05 05:50:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2319(8165273) in day on r815u03n07 Nov 05 05:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_256(7572424) WEXITSTATUS 0 Nov 05 05:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_256(7572424) done Nov 05 05:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_731(7572900) WEXITSTATUS 0 Nov 05 05:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_731(7572900) done Nov 05 05:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_516(7572684) WEXITSTATUS 0 Nov 05 05:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_516(7572684) done Nov 05 05:50:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144150 in scavenge on r907u38n01,r907u40n01 Nov 05 05:50:58 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6433(8164263) Nov 05 05:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_33(8165274) NodeList=r908u39n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6955(8165275) in scavenge on r801u06n01 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6956(8165276) in scavenge on r802u09n03 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6957(8165277) in scavenge on r802u11n01 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6958(8165278) in scavenge on r913u01n04 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6959(8165279) in scavenge on r914u05n04 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6960(8165280) in scavenge on r914u05n04 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6961(8165281) in scavenge on r805u06n04 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6962(8165282) in scavenge on r805u14n03 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6963(8165283) in scavenge on r918u05n04 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6964(8165284) in scavenge on r919u05n01 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6965(8165285) in scavenge on r919u05n01 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6966(8165286) in scavenge on r919u05n03 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6967(8165287) in scavenge on r919u09n01 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6968(8165288) in scavenge on r919u09n01 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6969(8165289) in scavenge on r919u09n02 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144120 WEXITSTATUS 0 Nov 05 05:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144120 done Nov 05 05:51:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_34(8165290) NodeList=r914u09n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:51:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_35(8165291) NodeList=r914u09n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:51:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_580(7572749) in scavenge on r816u09n03 Nov 05 05:51:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_20(8158125) WEXITSTATUS 1 Nov 05 05:51:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_20(8158125) done Nov 05 05:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_108(8163500) WEXITSTATUS 0 Nov 05 05:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_108(8163500) done Nov 05 05:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3817(8165175) WEXITSTATUS 0 Nov 05 05:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3817(8165175) done Nov 05 05:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6666(8164752) WEXITSTATUS 1 Nov 05 05:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6666(8164752) done Nov 05 05:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1507(8142948) WEXITSTATUS 0 Nov 05 05:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1507(8142948) done Nov 05 05:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_123(8163515) WEXITSTATUS 0 Nov 05 05:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_123(8163515) done Nov 05 05:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1794(8158105) WEXITSTATUS 0 Nov 05 05:51:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1794(8158105) done Nov 05 05:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1835(8158549) WEXITSTATUS 0 Nov 05 05:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1835(8158549) done Nov 05 05:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_56(8163390) WEXITSTATUS 0 Nov 05 05:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_56(8163390) done Nov 05 05:51:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_92(8163466) WEXITSTATUS 0 Nov 05 05:51:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_92(8163466) done Nov 05 05:51:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_36(8165292) NodeList=r807u08n04 #CPUs=16 Partition=day Nov 05 05:51:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_37(8165293) NodeList=r815u15n08 #CPUs=16 Partition=day Nov 05 05:51:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_38(8165294) NodeList=r815u23n02 #CPUs=16 Partition=day Nov 05 05:51:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_39(8165295) NodeList=r816u15n08 #CPUs=16 Partition=day Nov 05 05:51:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144132 NodeList=r807u01n01,r904u01n04 #CPUs=84 Partition=scavenge Nov 05 05:51:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_296(7572464) WEXITSTATUS 0 Nov 05 05:51:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_296(7572464) done Nov 05 05:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1850(8159082) WEXITSTATUS 0 Nov 05 05:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1850(8159082) done Nov 05 05:51:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_124(8163516) WEXITSTATUS 0 Nov 05 05:51:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_124(8163516) done Nov 05 05:51:49 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8144122 had to be killed Nov 05 05:51:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5960(8162960) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:51:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6461(8164366) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:51:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4581(8156094) WEXITSTATUS 1 Nov 05 05:51:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4581(8156094) done Nov 05 05:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_40(8165296) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_41(8165297) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_42(8165298) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:51:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6461(8164366) Nov 05 05:51:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5960(8162960) Nov 05 05:51:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_43(8165299) NodeList=r908u35n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:51:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_73(8163433) WEXITSTATUS 0 Nov 05 05:51:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_73(8163433) done Nov 05 05:51:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6957(8165277) WEXITSTATUS 1 Nov 05 05:51:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6957(8165277) done Nov 05 05:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1839(8158618) WEXITSTATUS 0 Nov 05 05:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1839(8158618) done Nov 05 05:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_168(8163584) WEXITSTATUS 0 Nov 05 05:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_168(8163584) done Nov 05 05:52:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1769(8157937) WEXITSTATUS 0 Nov 05 05:52:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1769(8157937) done Nov 05 05:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2405(8152449) WEXITSTATUS 0 Nov 05 05:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2405(8152449) done Nov 05 05:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2515(8152673) WEXITSTATUS 0 Nov 05 05:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2515(8152673) done Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:52:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6298(8163651) WEXITSTATUS 1 Nov 05 05:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6298(8163651) done Nov 05 05:52:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1766(8157872) WEXITSTATUS 0 Nov 05 05:52:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1766(8157872) done Nov 05 05:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_128(8163520) WEXITSTATUS 0 Nov 05 05:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_128(8163520) done Nov 05 05:52:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_132(8163524) WEXITSTATUS 0 Nov 05 05:52:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_132(8163524) done Nov 05 05:52:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_44(8165300) NodeList=r908u22n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:52:22 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6169(8163255) has been requeued to reclaim resources for JobId=8165203_45(8165203) Nov 05 05:52:22 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6581(8164621) has been requeued to reclaim resources for JobId=8165203_45(8165203) Nov 05 05:52:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6581(8164621) Nov 05 05:52:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6169(8163255) Nov 05 05:52:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_45(8165301) NodeList=r908u35n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:52:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1600(8153018) WEXITSTATUS 0 Nov 05 05:52:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1600(8153018) done Nov 05 05:52:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5509(8160512) WEXITSTATUS 1 Nov 05 05:52:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5509(8160512) done Nov 05 05:52:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_46(8165302) NodeList=r815u09n04 #CPUs=16 Partition=day Nov 05 05:52:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_47(8165303) NodeList=r815u09n06 #CPUs=16 Partition=day Nov 05 05:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_461(7572629) WEXITSTATUS 0 Nov 05 05:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_461(7572629) done Nov 05 05:52:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1749(8157766) WEXITSTATUS 0 Nov 05 05:52:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1749(8157766) done Nov 05 05:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_130(8163522) WEXITSTATUS 0 Nov 05 05:52:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_130(8163522) done Nov 05 05:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_127(8163519) WEXITSTATUS 0 Nov 05 05:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_127(8163519) done Nov 05 05:52:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6939(8165222) WEXITSTATUS 1 Nov 05 05:52:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6939(8165222) done Nov 05 05:52:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5791(8162746) WEXITSTATUS 1 Nov 05 05:52:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5791(8162746) done Nov 05 05:52:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_195(8163630) WEXITSTATUS 0 Nov 05 05:52:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_195(8163630) done Nov 05 05:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_218(8163670) WEXITSTATUS 0 Nov 05 05:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_218(8163670) done Nov 05 05:53:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_215(8163667) WEXITSTATUS 0 Nov 05 05:53:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_215(8163667) done Nov 05 05:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_191(8163626) WEXITSTATUS 0 Nov 05 05:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_191(8163626) done Nov 05 05:53:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6685(8164787) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:53:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6685(8164787) Nov 05 05:53:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_48(8165304) NodeList=r908u33n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:53:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_122(8163514) WEXITSTATUS 0 Nov 05 05:53:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_122(8163514) done Nov 05 05:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1723(8157329) WEXITSTATUS 0 Nov 05 05:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1723(8157329) done Nov 05 05:53:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165203_49(8165203) NodeList=r918u36n03 #CPUs=16 Partition=day Nov 05 05:53:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_0(8165305) NodeList=r816u15n01 #CPUs=16 Partition=day Nov 05 05:53:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_1(8165306) NodeList=r816u15n02 #CPUs=16 Partition=day Nov 05 05:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_662(7571627) WEXITSTATUS 0 Nov 05 05:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_662(7571627) done Nov 05 05:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_513(7572681) WEXITSTATUS 0 Nov 05 05:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_513(7572681) done Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165262_2(8165307) in day on r913u09n04 Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3820(8165308) in gpu on r808u14n01 Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2320(8165309) in day on r806u02n04 Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2321(8165310) in day on r904u07n04 Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2322(8165311) in day on r909u07n03 Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2323(8165312) in day on r912u28n02 Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2324(8165313) in day on r912u32n03 Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2325(8165314) in day on r918u01n02 Nov 05 05:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2326(8165315) in day on r919u36n02 Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6751(8164882) WEXITSTATUS 1 Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6751(8164882) done Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_525(7572694) WEXITSTATUS 0 Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_525(7572694) done Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_855(8115124) WEXITSTATUS 0 Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_855(8115124) done Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_781(8115050) WEXITSTATUS 0 Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_781(8115050) done Nov 05 05:53:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6970(8165316) in scavenge on r801u32n02 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6971(8165317) in scavenge on r802u11n01 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6972(8165318) in scavenge on r904u01n03 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6973(8165319) in scavenge on r815u15n12 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6974(8165320) in scavenge on r906u28n01 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6975(8165321) in scavenge on r805u27n02 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6976(8165322) in scavenge on r806u02n02 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6977(8165323) in scavenge on r913u09n04 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6978(8165324) in scavenge on r919u36n01 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6979(8165325) in scavenge on r919u36n03 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6980(8165326) in scavenge on r815u15n04 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6981(8165327) in scavenge on r815u15n04 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6982(8165328) in scavenge on r816u35n03 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6983(8165329) in scavenge on r816u03n03 Nov 05 05:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_51(8115521) in scavenge on r815u23n02 Nov 05 05:53:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3806(8165028) WEXITSTATUS 0 Nov 05 05:53:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3806(8165028) done Nov 05 05:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820336 WEXITSTATUS 0 Nov 05 05:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820336 done Nov 05 05:53:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820367 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 05:54:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_193(8163628) WEXITSTATUS 0 Nov 05 05:54:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_193(8163628) done Nov 05 05:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_221(8163673) WEXITSTATUS 0 Nov 05 05:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_221(8163673) done Nov 05 05:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1767(8157873) WEXITSTATUS 0 Nov 05 05:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1767(8157873) done Nov 05 05:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6607(8164662) WEXITSTATUS 1 Nov 05 05:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6607(8164662) done Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:54:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6304(8163700) WEXITSTATUS 1 Nov 05 05:54:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6304(8163700) done Nov 05 05:54:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1774(8157958) WEXITSTATUS 0 Nov 05 05:54:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1774(8157958) done Nov 05 05:54:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144118 WEXITSTATUS 0 Nov 05 05:54:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144118 done Nov 05 05:54:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_299(8163785) WEXITSTATUS 0 Nov 05 05:54:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_299(8163785) done Nov 05 05:54:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6087(8163120) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:54:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6711(8164822) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6835(8165030) WEXITSTATUS 1 Nov 05 05:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6835(8165030) done Nov 05 05:54:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6087(8163120) Nov 05 05:54:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6711(8164822) Nov 05 05:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_3(8165330) NodeList=r908u33n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_299(8114565) WEXITSTATUS 0 Nov 05 05:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_299(8114565) done Nov 05 05:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_131(8163523) WEXITSTATUS 0 Nov 05 05:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_131(8163523) done Nov 05 05:54:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6979(8165325) WEXITSTATUS 1 Nov 05 05:54:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6979(8165325) done Nov 05 05:54:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_59(8163419) WEXITSTATUS 0 Nov 05 05:54:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_59(8163419) done Nov 05 05:54:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_199(8163634) WEXITSTATUS 0 Nov 05 05:54:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_199(8163634) done Nov 05 05:54:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_140(8163532) WEXITSTATUS 0 Nov 05 05:54:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_140(8163532) done Nov 05 05:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_4(8165331) NodeList=r815u09n06 #CPUs=16 Partition=day Nov 05 05:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_5(8165332) NodeList=r815u23n01 #CPUs=16 Partition=day Nov 05 05:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_6(8165333) NodeList=r815u35n05 #CPUs=16 Partition=day Nov 05 05:54:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144140 NodeList=r801u27n03,r916u01n02 #CPUs=84 Partition=scavenge Nov 05 05:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_571(7572740) WEXITSTATUS 0 Nov 05 05:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_571(7572740) done Nov 05 05:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_514(7572682) WEXITSTATUS 0 Nov 05 05:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_514(7572682) done Nov 05 05:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1662(8156295) WEXITSTATUS 0 Nov 05 05:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1662(8156295) done Nov 05 05:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_200(8163635) WEXITSTATUS 0 Nov 05 05:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_200(8163635) done Nov 05 05:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_196(8163631) WEXITSTATUS 0 Nov 05 05:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_196(8163631) done Nov 05 05:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1828(8158507) WEXITSTATUS 0 Nov 05 05:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1828(8158507) done Nov 05 05:54:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 05:55:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6906(8165161) WEXITSTATUS 1 Nov 05 05:55:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6906(8165161) done Nov 05 05:55:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1759(8157828) WEXITSTATUS 0 Nov 05 05:55:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1759(8157828) done Nov 05 05:55:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6713(8164824) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:55:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3820(8165308) WEXITSTATUS 0 Nov 05 05:55:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3820(8165308) done Nov 05 05:55:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6713(8164824) Nov 05 05:55:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_7(8165334) NodeList=r908u35n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8127999 WEXITSTATUS 0 Nov 05 05:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8127999 done Nov 05 05:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1876(8161847) WEXITSTATUS 0 Nov 05 05:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1876(8161847) done Nov 05 05:55:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3802(8164962) WEXITSTATUS 0 Nov 05 05:55:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3802(8164962) done Nov 05 05:55:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1815(8158427) WEXITSTATUS 0 Nov 05 05:55:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1815(8158427) done Nov 05 05:55:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_8(8165335) NodeList=r805u06n02 #CPUs=16 Partition=day Nov 05 05:55:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_9(8165336) NodeList=r918u01n04 #CPUs=16 Partition=day Nov 05 05:55:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_10(8165337) NodeList=r918u01n04 #CPUs=16 Partition=day Nov 05 05:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_670(7572839) WEXITSTATUS 0 Nov 05 05:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_670(7572839) done Nov 05 05:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_156(8163572) WEXITSTATUS 0 Nov 05 05:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_156(8163572) done Nov 05 05:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_110(8163502) WEXITSTATUS 0 Nov 05 05:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_110(8163502) done Nov 05 05:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154628 WEXITSTATUS 0 Nov 05 05:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154628 done Nov 05 05:55:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_101(8163475) WEXITSTATUS 0 Nov 05 05:55:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_101(8163475) done Nov 05 05:55:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165338 InitPrio=37738 usec=3773 Nov 05 05:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6156(8163203) WEXITSTATUS 1 Nov 05 05:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6156(8163203) done Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:56:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_146(8163562) WEXITSTATUS 0 Nov 05 05:56:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_146(8163562) done Nov 05 05:56:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5163(8158517) WEXITSTATUS 1 Nov 05 05:56:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5163(8158517) done Nov 05 05:56:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6774(8164926) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:56:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6774(8164926) Nov 05 05:56:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_11(8165339) NodeList=r908u39n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:56:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3821(8165340) in gpu on r808u14n01 Nov 05 05:56:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3822(8165341) in gpu on r808u14n01 Nov 05 05:56:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3823(8165342) in gpu on r4513u20n01 Nov 05 05:56:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154718 in day on r816u03n08 Nov 05 05:56:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2327(8165343) in day on r807u08n02 Nov 05 05:56:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2328(8165344) in day on r909u13n03 Nov 05 05:56:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2329(8165345) in day on r913u09n01 Nov 05 05:56:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2330(8165346) in day on r914u05n04 Nov 05 05:56:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2331(8165347) in day on r918u32n02 Nov 05 05:56:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2332(8165348) in day on r815u35n05 Nov 05 05:56:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6267(8163549) WEXITSTATUS 1 Nov 05 05:56:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6267(8163549) done Nov 05 05:56:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3821(8165340) WEXITSTATUS 0 Nov 05 05:56:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3821(8165340) done Nov 05 05:56:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133197 WEXITSTATUS 0 Nov 05 05:56:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133197 done Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6984(8165349) in scavenge on r802u06n02 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6985(8165350) in scavenge on r802u11n01 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6986(8165351) in scavenge on r815u15n12 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6987(8165352) in scavenge on r815u23n07 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6988(8165353) in scavenge on r805u06n02 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6989(8165354) in scavenge on r806u02n03 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6990(8165355) in scavenge on r807u05n04 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6991(8165356) in scavenge on r913u28n02 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6992(8165357) in scavenge on r918u01n04 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6993(8165358) in scavenge on r815u09n06 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6994(8165359) in scavenge on r815u23n03 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165262_12(8165360) in scavenge on r816u03n12 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_263(8163733) WEXITSTATUS 0 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_263(8163733) done Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6468(8164373) WEXITSTATUS 1 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6468(8164373) done Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_171(8163587) WEXITSTATUS 0 Nov 05 05:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_171(8163587) done Nov 05 05:56:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4078(8148345) in scavenge on r815u23n03 Nov 05 05:56:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_544(8116017) in scavenge on r917u13n04 Nov 05 05:56:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_648(7571613) in scavenge on r911u32n01 Nov 05 05:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6599(8164641) WEXITSTATUS 1 Nov 05 05:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6599(8164641) done Nov 05 05:56:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1834(8158548) WEXITSTATUS 0 Nov 05 05:56:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1834(8158548) done Nov 05 05:56:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_100(8163474) WEXITSTATUS 0 Nov 05 05:56:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_100(8163474) done Nov 05 05:56:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_222(8163674) WEXITSTATUS 0 Nov 05 05:56:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_222(8163674) done Nov 05 05:56:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1768(8157874) WEXITSTATUS 0 Nov 05 05:56:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1768(8157874) done Nov 05 05:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1681(8156314) WEXITSTATUS 0 Nov 05 05:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1681(8156314) done Nov 05 05:57:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_213(8163665) WEXITSTATUS 0 Nov 05 05:57:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_213(8163665) done Nov 05 05:57:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6418(8164225) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:57:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6782(8164936) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_175(8163610) WEXITSTATUS 0 Nov 05 05:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_175(8163610) done Nov 05 05:57:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6418(8164225) Nov 05 05:57:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6782(8164936) Nov 05 05:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3823(8165342) WEXITSTATUS 0 Nov 05 05:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3823(8165342) done Nov 05 05:57:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_13(8165361) NodeList=r908u33n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:57:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_135(8163527) WEXITSTATUS 0 Nov 05 05:57:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_135(8163527) done Nov 05 05:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_106(8163480) WEXITSTATUS 0 Nov 05 05:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_106(8163480) done Nov 05 05:57:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6948(8165240) WEXITSTATUS 1 Nov 05 05:57:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6948(8165240) done Nov 05 05:57:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_72(8163432) WEXITSTATUS 0 Nov 05 05:57:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_72(8163432) done Nov 05 05:57:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_230(8163682) WEXITSTATUS 0 Nov 05 05:57:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_230(8163682) done Nov 05 05:57:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5699(8162361) WEXITSTATUS 1 Nov 05 05:57:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5699(8162361) done Nov 05 05:57:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_99(8163473) WEXITSTATUS 0 Nov 05 05:57:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_99(8163473) done Nov 05 05:57:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_223(8163675) WEXITSTATUS 0 Nov 05 05:57:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_223(8163675) done Nov 05 05:57:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_170(8163586) WEXITSTATUS 0 Nov 05 05:57:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_170(8163586) done Nov 05 05:57:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6699(8164804) WEXITSTATUS 1 Nov 05 05:57:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6699(8164804) done Nov 05 05:57:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3753(8164409) WEXITSTATUS 0 Nov 05 05:57:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3753(8164409) done Nov 05 05:57:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_293(8163779) WEXITSTATUS 0 Nov 05 05:57:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_293(8163779) done Nov 05 05:58:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_224(8163676) WEXITSTATUS 0 Nov 05 05:58:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_224(8163676) done Nov 05 05:58:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8165338 uid 10575 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 05:58:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 05:58:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_160(8163576) WEXITSTATUS 0 Nov 05 05:58:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_160(8163576) done Nov 05 05:58:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6492(8164416) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:58:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6826(8165013) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:58:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165362 InitPrio=37737 usec=4168 Nov 05 05:58:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6492(8164416) Nov 05 05:58:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6826(8165013) Nov 05 05:58:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_14(8165363) NodeList=r908u33n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_133(8163525) WEXITSTATUS 0 Nov 05 05:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_133(8163525) done Nov 05 05:58:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_53(8135752) WEXITSTATUS 0 Nov 05 05:58:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_53(8135752) done Nov 05 05:58:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_682(7572851) WEXITSTATUS 0 Nov 05 05:58:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_682(7572851) done Nov 05 05:58:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_692(7572861) WEXITSTATUS 0 Nov 05 05:58:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_692(7572861) done Nov 05 05:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_317(8163818) WEXITSTATUS 0 Nov 05 05:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_317(8163818) done Nov 05 05:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1659(8156292) WEXITSTATUS 0 Nov 05 05:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1659(8156292) done Nov 05 05:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_214(8163666) WEXITSTATUS 0 Nov 05 05:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_214(8163666) done Nov 05 05:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5625(8161745) WEXITSTATUS 1 Nov 05 05:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5625(8161745) done Nov 05 05:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6982(8165328) WEXITSTATUS 1 Nov 05 05:59:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6982(8165328) done Nov 05 05:59:14 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6084(8163117) has been requeued to reclaim resources for JobId=8165262_15(8165262) Nov 05 05:59:14 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6865(8165088) has been requeued to reclaim resources for JobId=8165262_15(8165262) Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3824(8165364) in gpu on r808u08n01 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3825(8165365) in gpu on r808u14n01 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3826(8165366) in gpu on r4513u20n01 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2333(8165367) in day on r807u11n04 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2334(8165368) in day on r909u13n04 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2335(8165369) in day on r904u04n03 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2336(8165370) in day on r815u29n04 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144125 WEXITSTATUS 0 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144125 done Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144128 WEXITSTATUS 0 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144128 done Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_162(8163578) WEXITSTATUS 0 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_162(8163578) done Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144124 WEXITSTATUS 0 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144124 done Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144126 WEXITSTATUS 0 Nov 05 05:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144126 done Nov 05 05:59:16 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6865(8165088) Nov 05 05:59:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6084(8163117) Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_15(8165371) NodeList=r908u39n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_16(8165372) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_17(8165373) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_18(8165374) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_19(8165375) NodeList=r915u29n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_20(8165376) NodeList=r915u29n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_21(8165377) NodeList=r915u29n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_22(8165378) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_23(8165379) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_24(8165380) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_25(8165381) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_26(8165382) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_27(8165383) NodeList=r914u13n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_28(8165384) NodeList=r914u13n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_29(8165385) NodeList=r914u13n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_30(8165386) NodeList=r914u13n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_31(8165387) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_32(8165388) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6995(8165389) in scavenge on r907u05n01 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6996(8165390) in scavenge on r802u11n02 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6997(8165391) in scavenge on r816u29n08 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6998(8165392) in scavenge on r903u26n01 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_6999(8165393) in scavenge on r806u09n03 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7000(8165394) in scavenge on r904u04n03 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7001(8165395) in scavenge on r918u32n04 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7002(8165396) in scavenge on r805u04n01 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7003(8165397) in scavenge on r805u04n02 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7004(8165398) in scavenge on r816u35n03 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7005(8165399) in scavenge on r815u29n01 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7006(8165400) in scavenge on r815u29n03 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_57(8114323) in scavenge on r903u12n01 Nov 05 05:59:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_112(8114378) in scavenge on r913u32n01 Nov 05 05:59:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_102(8163476) WEXITSTATUS 0 Nov 05 05:59:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_102(8163476) done Nov 05 05:59:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_104(8163478) WEXITSTATUS 0 Nov 05 05:59:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_104(8163478) done Nov 05 05:59:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_165(8163581) WEXITSTATUS 0 Nov 05 05:59:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_165(8163581) done Nov 05 05:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165362 NodeList=r815u09n05 #CPUs=36 Partition=day Nov 05 05:59:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7003(8165397) WEXITSTATUS 1 Nov 05 05:59:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7003(8165397) done Nov 05 05:59:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6171(8163257) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:59:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6887(8165126) has been requeued to reclaim resources for JobId=Invalid Nov 05 05:59:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_368(8163889) WEXITSTATUS 0 Nov 05 05:59:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_368(8163889) done Nov 05 05:59:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_227(8163679) WEXITSTATUS 0 Nov 05 05:59:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_227(8163679) done Nov 05 05:59:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6171(8163257) Nov 05 05:59:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6887(8165126) Nov 05 05:59:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_33(8165401) NodeList=r908u39n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144129 WEXITSTATUS 0 Nov 05 05:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144129 done Nov 05 05:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_232(8163684) WEXITSTATUS 0 Nov 05 05:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_232(8163684) done Nov 05 05:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3824(8165364) WEXITSTATUS 0 Nov 05 05:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3824(8165364) done Nov 05 05:59:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_34(8165402) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_35(8165403) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_36(8165404) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_37(8165405) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_38(8165406) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 05:59:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_39(8165407) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 06:00:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6912(8165167) WEXITSTATUS 1 Nov 05 06:00:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6912(8165167) done Nov 05 06:00:02 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 06:00:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 06:00:02 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 06:00:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 06:00:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_129(8163521) WEXITSTATUS 0 Nov 05 06:00:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_129(8163521) done Nov 05 06:00:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1720(8157214) WEXITSTATUS 0 Nov 05 06:00:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1720(8157214) done Nov 05 06:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_103(8163477) WEXITSTATUS 0 Nov 05 06:00:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_103(8163477) done Nov 05 06:00:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:00:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_233(8163685) WEXITSTATUS 0 Nov 05 06:00:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_233(8163685) done Nov 05 06:00:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 06:00:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 06:00:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_771(7572940) WEXITSTATUS 0 Nov 05 06:00:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_771(7572940) done Nov 05 06:00:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_893(7571859) WEXITSTATUS 0 Nov 05 06:00:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_893(7571859) done Nov 05 06:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_177(8163612) WEXITSTATUS 0 Nov 05 06:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_177(8163612) done Nov 05 06:00:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 06:00:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 06:00:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 06:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5938(8162933) WEXITSTATUS 1 Nov 05 06:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5938(8162933) done Nov 05 06:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_220(8163672) WEXITSTATUS 0 Nov 05 06:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_220(8163672) done Nov 05 06:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_155(8163571) WEXITSTATUS 0 Nov 05 06:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_155(8163571) done Nov 05 06:01:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_197(8163632) WEXITSTATUS 0 Nov 05 06:01:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_197(8163632) done Nov 05 06:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4922(8158109) WEXITSTATUS 1 Nov 05 06:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4922(8158109) done Nov 05 06:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6673(8164759) WEXITSTATUS 1 Nov 05 06:01:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6673(8164759) done Nov 05 06:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3797(8164911) WEXITSTATUS 0 Nov 05 06:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3797(8164911) done Nov 05 06:01:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_126(8163518) WEXITSTATUS 0 Nov 05 06:01:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_126(8163518) done Nov 05 06:01:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_40(8165408) NodeList=r815u09n03 #CPUs=16 Partition=day Nov 05 06:01:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_914(7573083) WEXITSTATUS 0 Nov 05 06:01:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_914(7573083) done Nov 05 06:01:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6968(8165288) WEXITSTATUS 1 Nov 05 06:01:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6968(8165288) done Nov 05 06:01:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_292(8163778) WEXITSTATUS 0 Nov 05 06:01:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_292(8163778) done Nov 05 06:01:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_105(8163479) WEXITSTATUS 0 Nov 05 06:01:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_105(8163479) done Nov 05 06:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_134(8163526) WEXITSTATUS 0 Nov 05 06:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_134(8163526) done Nov 05 06:01:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3827(8165409) in gpu on r808u08n01 Nov 05 06:01:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3828(8165410) in gpu on r808u17n01 Nov 05 06:01:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2337(8165411) in day on r919u05n04 Nov 05 06:02:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_173(8163608) WEXITSTATUS 0 Nov 05 06:02:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_173(8163608) done Nov 05 06:02:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4078(8148345) WEXITSTATUS 1 Nov 05 06:02:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4078(8148345) done Nov 05 06:02:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_164(8163580) WEXITSTATUS 0 Nov 05 06:02:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_164(8163580) done Nov 05 06:02:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1845(8158682) WEXITSTATUS 0 Nov 05 06:02:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1845(8158682) done Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7007(8165412) in scavenge on r907u09n03 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7008(8165413) in scavenge on r805u04n02 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7009(8165414) in scavenge on r913u09n02 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7010(8165415) in scavenge on r919u09n01 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7011(8165416) in scavenge on r919u36n03 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7012(8165417) in scavenge on r909u04n04 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7013(8165418) in scavenge on r909u13n01 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7014(8165419) in scavenge on r909u13n02 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7015(8165420) in scavenge on r815u03n07 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165262_41(8165421) in scavenge on r815u35n04 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_217(8114483) in scavenge on r807u22n01 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_831(7573000) in scavenge on r903u24n01 Nov 05 06:02:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_866(7573035) in scavenge on r913u35n01 Nov 05 06:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3825(8165365) WEXITSTATUS 0 Nov 05 06:02:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3825(8165365) done Nov 05 06:02:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6946(8165238) WEXITSTATUS 1 Nov 05 06:02:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6946(8165238) done Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:02:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_154(8163570) WEXITSTATUS 0 Nov 05 06:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_154(8163570) done Nov 05 06:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_98(8163472) WEXITSTATUS 0 Nov 05 06:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_98(8163472) done Nov 05 06:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1848(8158851) WEXITSTATUS 0 Nov 05 06:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1848(8158851) done Nov 05 06:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_899(8128061) WEXITSTATUS 0 Nov 05 06:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_899(8128061) done Nov 05 06:02:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_297(8163783) WEXITSTATUS 0 Nov 05 06:02:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_297(8163783) done Nov 05 06:02:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_799(7572968) WEXITSTATUS 0 Nov 05 06:02:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_799(7572968) done Nov 05 06:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6246(8163452) WEXITSTATUS 1 Nov 05 06:02:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6246(8163452) done Nov 05 06:02:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6649(8164728) WEXITSTATUS 1 Nov 05 06:02:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6649(8164728) done Nov 05 06:02:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3828(8165410) WEXITSTATUS 0 Nov 05 06:02:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3828(8165410) done Nov 05 06:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3827(8165409) WEXITSTATUS 0 Nov 05 06:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3827(8165409) done Nov 05 06:02:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907439 WEXITSTATUS 0 Nov 05 06:02:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907439 done Nov 05 06:02:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6674(8164760) WEXITSTATUS 1 Nov 05 06:02:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6674(8164760) done Nov 05 06:03:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1762(8157844) WEXITSTATUS 0 Nov 05 06:03:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1762(8157844) done Nov 05 06:03:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_219(8163671) WEXITSTATUS 0 Nov 05 06:03:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_219(8163671) done Nov 05 06:03:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 06:03:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 06:03:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 06:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6852(8165061) WEXITSTATUS 1 Nov 05 06:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6852(8165061) done Nov 05 06:03:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1777(8157984) WEXITSTATUS 0 Nov 05 06:03:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1777(8157984) done Nov 05 06:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_136(8163528) WEXITSTATUS 0 Nov 05 06:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_136(8163528) done Nov 05 06:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_366(8163887) WEXITSTATUS 0 Nov 05 06:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_366(8163887) done Nov 05 06:03:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_186(8163621) WEXITSTATUS 0 Nov 05 06:03:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_186(8163621) done Nov 05 06:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_159(8163575) WEXITSTATUS 0 Nov 05 06:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_159(8163575) done Nov 05 06:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_201(8163636) WEXITSTATUS 0 Nov 05 06:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_201(8163636) done Nov 05 06:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1680(8156313) WEXITSTATUS 0 Nov 05 06:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1680(8156313) done Nov 05 06:03:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_42(8165422) NodeList=r909u29n02 #CPUs=16 Partition=day Nov 05 06:03:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_43(8165423) NodeList=r815u23n03 #CPUs=16 Partition=day Nov 05 06:03:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_656(7572825) WEXITSTATUS 0 Nov 05 06:03:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_656(7572825) done Nov 05 06:03:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_163(8163579) WEXITSTATUS 0 Nov 05 06:03:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_163(8163579) done Nov 05 06:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1814(8158426) WEXITSTATUS 0 Nov 05 06:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1814(8158426) done Nov 05 06:03:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1521(8144001) WEXITSTATUS 0 Nov 05 06:03:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1521(8144001) done Nov 05 06:04:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_194(8163629) WEXITSTATUS 0 Nov 05 06:04:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_194(8163629) done Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_332(8163833) WEXITSTATUS 0 Nov 05 06:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_332(8163833) done Nov 05 06:04:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1745(8157762) WEXITSTATUS 0 Nov 05 06:04:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1745(8157762) done Nov 05 06:04:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_264(8163734) WEXITSTATUS 0 Nov 05 06:04:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_264(8163734) done Nov 05 06:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_361(8163882) WEXITSTATUS 0 Nov 05 06:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_361(8163882) done Nov 05 06:04:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1149(8132555) WEXITSTATUS 0 Nov 05 06:04:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1149(8132555) done Nov 05 06:04:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6993(8165358) WEXITSTATUS 1 Nov 05 06:04:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6993(8165358) done Nov 05 06:04:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_142(8163558) WEXITSTATUS 0 Nov 05 06:04:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_142(8163558) done Nov 05 06:04:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_44(8165424) NodeList=r912u32n01 #CPUs=16 Partition=day Nov 05 06:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_972(7573141) WEXITSTATUS 0 Nov 05 06:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_972(7573141) done Nov 05 06:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_8(8114274) WEXITSTATUS 0 Nov 05 06:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_8(8114274) done Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165262_45(8165425) in day on r918u01n02 Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907440 in gpu on r805u38n01 Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3829(8165426) in gpu on r808u08n01 Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3830(8165427) in gpu on r808u14n01 Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3831(8165428) in gpu on r808u17n01 Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2338(8165429) in day on r807u05n01 Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2339(8165430) in day on r913u05n04 Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2340(8165431) in day on r919u01n04 Nov 05 06:04:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2341(8165432) in day on r815u29n02 Nov 05 06:04:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:04:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7016(8165433) in scavenge on r907u01n02 Nov 05 06:04:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7017(8165434) in scavenge on r802u06n03 Nov 05 06:04:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7018(8165435) in scavenge on r815u29n12 Nov 05 06:04:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7019(8165436) in scavenge on r807u24n01 Nov 05 06:04:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7020(8165437) in scavenge on r815u09n06 Nov 05 06:04:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7021(8165438) in scavenge on r815u29n03 Nov 05 06:04:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_874(7573043) in scavenge on r806u35n01 Nov 05 06:04:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_924(7573093) in scavenge on r806u35n01 Nov 05 06:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_464(8164017) WEXITSTATUS 0 Nov 05 06:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_464(8164017) done Nov 05 06:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5986(8162986) WEXITSTATUS 1 Nov 05 06:04:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5986(8162986) done Nov 05 06:04:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2675(8153062) WEXITSTATUS 0 Nov 05 06:04:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2675(8153062) done Nov 05 06:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_225(8163677) WEXITSTATUS 0 Nov 05 06:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_225(8163677) done Nov 05 06:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6928(8165200) WEXITSTATUS 1 Nov 05 06:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6928(8165200) done Nov 05 06:05:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_370(8163891) WEXITSTATUS 0 Nov 05 06:05:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_370(8163891) done Nov 05 06:05:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7017(8165434) WEXITSTATUS 1 Nov 05 06:05:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7017(8165434) done Nov 05 06:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_231(8163683) WEXITSTATUS 0 Nov 05 06:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_231(8163683) done Nov 05 06:05:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6650(8164729) WEXITSTATUS 1 Nov 05 06:05:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6650(8164729) done Nov 05 06:05:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_295(8163781) WEXITSTATUS 0 Nov 05 06:05:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_295(8163781) done Nov 05 06:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5710(8162423) WEXITSTATUS 1 Nov 05 06:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5710(8162423) done Nov 05 06:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_324(8163825) WEXITSTATUS 0 Nov 05 06:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_324(8163825) done Nov 05 06:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820335 WEXITSTATUS 0 Nov 05 06:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820335 done Nov 05 06:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7007(8165412) WEXITSTATUS 1 Nov 05 06:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7007(8165412) done Nov 05 06:05:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820368 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 06:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_12(8114278) WEXITSTATUS 0 Nov 05 06:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_12(8114278) done Nov 05 06:05:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_226(8163678) WEXITSTATUS 0 Nov 05 06:05:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_226(8163678) done Nov 05 06:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1738(8157741) WEXITSTATUS 0 Nov 05 06:05:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1738(8157741) done Nov 05 06:05:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1846(8158683) WEXITSTATUS 0 Nov 05 06:05:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1846(8158683) done Nov 05 06:05:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_188(8163623) WEXITSTATUS 0 Nov 05 06:06:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_188(8163623) done Nov 05 06:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6543(8164493) WEXITSTATUS 1 Nov 05 06:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6543(8164493) done Nov 05 06:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_261(8163731) WEXITSTATUS 0 Nov 05 06:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_261(8163731) done Nov 05 06:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5661(8162028) WEXITSTATUS 1 Nov 05 06:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5661(8162028) done Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:06:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_265(8163735) WEXITSTATUS 0 Nov 05 06:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_265(8163735) done Nov 05 06:06:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_408(8163941) WEXITSTATUS 0 Nov 05 06:06:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_408(8163941) done Nov 05 06:06:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:06:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_369(8163890) WEXITSTATUS 0 Nov 05 06:06:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_369(8163890) done Nov 05 06:06:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6898(8165145) WEXITSTATUS 1 Nov 05 06:06:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6898(8165145) done Nov 05 06:06:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_410(8163943) WEXITSTATUS 0 Nov 05 06:06:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_410(8163943) done Nov 05 06:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_189(8163624) WEXITSTATUS 0 Nov 05 06:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_189(8163624) done Nov 05 06:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_253(8163723) WEXITSTATUS 0 Nov 05 06:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_253(8163723) done Nov 05 06:06:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6361(8163952) WEXITSTATUS 1 Nov 05 06:06:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6361(8163952) done Nov 05 06:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6735(8164850) WEXITSTATUS 1 Nov 05 06:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6735(8164850) done Nov 05 06:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6332(8163796) WEXITSTATUS 1 Nov 05 06:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6332(8163796) done Nov 05 06:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6970(8165316) WEXITSTATUS 1 Nov 05 06:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6970(8165316) done Nov 05 06:06:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165262_46(8165439) NodeList=r815u15n03 #CPUs=16 Partition=day Nov 05 06:06:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_877(7573046) WEXITSTATUS 0 Nov 05 06:06:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_877(7573046) done Nov 05 06:06:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1695(8156717) WEXITSTATUS 0 Nov 05 06:06:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1695(8156717) done Nov 05 06:06:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1726(8157676) WEXITSTATUS 0 Nov 05 06:06:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1726(8157676) done Nov 05 06:06:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1697(8156719) WEXITSTATUS 0 Nov 05 06:06:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1697(8156719) done Nov 05 06:06:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_234(8163686) WEXITSTATUS 0 Nov 05 06:06:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_234(8163686) done Nov 05 06:06:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1860(8160127) WEXITSTATUS 0 Nov 05 06:06:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1860(8160127) done Nov 05 06:07:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5439(8159654) WEXITSTATUS 1 Nov 05 06:07:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5439(8159654) done Nov 05 06:07:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_183(8163618) WEXITSTATUS 0 Nov 05 06:07:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_183(8163618) done Nov 05 06:07:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_316(8163817) WEXITSTATUS 0 Nov 05 06:07:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_316(8163817) done Nov 05 06:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6558(8164580) WEXITSTATUS 1 Nov 05 06:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6558(8164580) done Nov 05 06:07:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165440 InitPrio=20041 usec=3833 Nov 05 06:07:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165173 WEXITSTATUS 0 Nov 05 06:07:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165173 done Nov 05 06:07:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3812(8165087) WEXITSTATUS 0 Nov 05 06:07:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3812(8165087) done Nov 05 06:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_287(8163773) WEXITSTATUS 0 Nov 05 06:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_287(8163773) done Nov 05 06:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1696(8156718) WEXITSTATUS 0 Nov 05 06:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1696(8156718) done Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165262_47(8165441) in day on r807u11n03 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165262_48(8165442) in day on r919u32n03 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165262_49(8165262) in day on r919u32n03 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165440 in day on r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3832(8165443) in gpu on r4513u23n01 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2342(8165444) in day on r904u04n02 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2343(8165445) in day on r815u35n05 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2344(8165446) in day on r816u03n02 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_681(8114950) WEXITSTATUS 0 Nov 05 06:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_681(8114950) done Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7022(8165447) in scavenge on r908u28n02 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7023(8165448) in scavenge on r907u09n03 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7024(8165449) in scavenge on r801u32n02 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7025(8165450) in scavenge on r802u06n03 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7026(8165451) in scavenge on r916u01n01 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7027(8165452) in scavenge on r815u29n11 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7028(8165453) in scavenge on r903u34n01 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7029(8165454) in scavenge on r805u09n01 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7030(8165455) in scavenge on r909u10n04 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7031(8165456) in scavenge on r912u36n02 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7032(8165457) in scavenge on r914u05n02 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7033(8165458) in scavenge on r918u05n03 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7034(8165459) in scavenge on r918u32n02 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7035(8165460) in scavenge on r919u01n01 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7036(8165461) in scavenge on r815u29n03 Nov 05 06:07:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_138(8114404) in scavenge on r911u32n01 Nov 05 06:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_184(8163619) WEXITSTATUS 0 Nov 05 06:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_184(8163619) done Nov 05 06:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7002(8165396) WEXITSTATUS 1 Nov 05 06:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7002(8165396) done Nov 05 06:07:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_179(8163614) WEXITSTATUS 0 Nov 05 06:07:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_179(8163614) done Nov 05 06:07:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2345(8165462) NodeList=r919u32n03 #CPUs=1 Partition=day Nov 05 06:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_251(8163721) WEXITSTATUS 0 Nov 05 06:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_251(8163721) done Nov 05 06:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_331(8163832) WEXITSTATUS 0 Nov 05 06:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_331(8163832) done Nov 05 06:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_386(8163919) WEXITSTATUS 0 Nov 05 06:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_386(8163919) done Nov 05 06:07:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_228(8163680) WEXITSTATUS 0 Nov 05 06:07:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_228(8163680) done Nov 05 06:07:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165463 InitPrio=20705 usec=3728 Nov 05 06:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_145(8163561) WEXITSTATUS 0 Nov 05 06:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_145(8163561) done Nov 05 06:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_185(8163620) WEXITSTATUS 0 Nov 05 06:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_185(8163620) done Nov 05 06:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_431(8163974) WEXITSTATUS 0 Nov 05 06:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_431(8163974) done Nov 05 06:08:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_256(8163726) WEXITSTATUS 0 Nov 05 06:08:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_256(8163726) done Nov 05 06:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6272(8163554) WEXITSTATUS 1 Nov 05 06:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6272(8163554) done Nov 05 06:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_390(8163923) WEXITSTATUS 0 Nov 05 06:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_390(8163923) done Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1756(8157798) WEXITSTATUS 0 Nov 05 06:08:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1756(8157798) done Nov 05 06:08:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_291(8163777) WEXITSTATUS 0 Nov 05 06:08:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_291(8163777) done Nov 05 06:08:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6550(8164503) WEXITSTATUS 1 Nov 05 06:08:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6550(8164503) done Nov 05 06:08:21 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1846(8124773) has been requeued to reclaim resources for JobId=Invalid Nov 05 06:08:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1846(8124773) Nov 05 06:08:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_157(8163573) WEXITSTATUS 0 Nov 05 06:08:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_157(8163573) done Nov 05 06:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_238(8163690) WEXITSTATUS 0 Nov 05 06:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_238(8163690) done Nov 05 06:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_253(8114519) WEXITSTATUS 0 Nov 05 06:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_253(8114519) done Nov 05 06:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_974(7573143) WEXITSTATUS 0 Nov 05 06:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_974(7573143) done Nov 05 06:08:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6985(8165350) WEXITSTATUS 1 Nov 05 06:08:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6985(8165350) done Nov 05 06:08:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6817(8164996) WEXITSTATUS 1 Nov 05 06:08:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6817(8164996) done Nov 05 06:08:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3829(8165426) WEXITSTATUS 0 Nov 05 06:08:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3829(8165426) done Nov 05 06:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_178(8163613) WEXITSTATUS 0 Nov 05 06:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_178(8163613) done Nov 05 06:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_328(8163829) WEXITSTATUS 0 Nov 05 06:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_328(8163829) done Nov 05 06:09:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1791(8158102) WEXITSTATUS 0 Nov 05 06:09:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1791(8158102) done Nov 05 06:09:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6831(8165018) WEXITSTATUS 1 Nov 05 06:09:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6831(8165018) done Nov 05 06:09:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_354(8163875) WEXITSTATUS 0 Nov 05 06:09:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_354(8163875) done Nov 05 06:09:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1690(8156712) WEXITSTATUS 0 Nov 05 06:09:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1690(8156712) done Nov 05 06:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2623(8127881) WEXITSTATUS 1 Nov 05 06:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2623(8127881) done Nov 05 06:09:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_0(8165464) NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 06:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_308(8114574) WEXITSTATUS 0 Nov 05 06:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_308(8114574) done Nov 05 06:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1006(7573175) WEXITSTATUS 0 Nov 05 06:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1006(7573175) done Nov 05 06:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_203(8163655) WEXITSTATUS 0 Nov 05 06:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_203(8163655) done Nov 05 06:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_172(8163607) WEXITSTATUS 0 Nov 05 06:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_172(8163607) done Nov 05 06:09:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6850(8165050) WEXITSTATUS 1 Nov 05 06:09:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6850(8165050) done Nov 05 06:09:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_143(8163559) WEXITSTATUS 0 Nov 05 06:09:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_143(8163559) done Nov 05 06:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_262(8163732) WEXITSTATUS 0 Nov 05 06:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_262(8163732) done Nov 05 06:10:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165463_1(8165465) in day on r909u07n02 Nov 05 06:10:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3833(8165466) in gpu on r808u08n01 Nov 05 06:10:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2346(8165467) in day on r807u11n03 Nov 05 06:10:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2347(8165468) in day on r919u05n04 Nov 05 06:10:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2348(8165469) in day on r909u07n01 Nov 05 06:10:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2349(8165470) in day on r909u13n04 Nov 05 06:10:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1747(8157764) WEXITSTATUS 0 Nov 05 06:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1747(8157764) done Nov 05 06:10:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_268(8163738) WEXITSTATUS 0 Nov 05 06:10:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_268(8163738) done Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7037(8165471) in scavenge on r801u06n03 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7038(8165472) in scavenge on r802u11n01 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7039(8165473) in scavenge on r917u13n04 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7040(8165474) in scavenge on r815u29n12 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7041(8165475) in scavenge on r806u23n01 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7042(8165476) in scavenge on r909u13n03 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7043(8165477) in scavenge on r914u01n02 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7044(8165478) in scavenge on r919u32n03 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7045(8165479) in scavenge on r805u04n01 Nov 05 06:10:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_794(7572963) in scavenge on r805u27n02 Nov 05 06:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_158(8163574) WEXITSTATUS 0 Nov 05 06:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_158(8163574) done Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:10:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:10:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1719(8157213) WEXITSTATUS 0 Nov 05 06:10:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1719(8157213) done Nov 05 06:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820337 WEXITSTATUS 0 Nov 05 06:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820337 done Nov 05 06:10:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820369 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 06:10:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_314(8114581) WEXITSTATUS 0 Nov 05 06:10:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_314(8114581) done Nov 05 06:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_260(8163730) WEXITSTATUS 0 Nov 05 06:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_260(8163730) done Nov 05 06:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7031(8165456) WEXITSTATUS 1 Nov 05 06:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7031(8165456) done Nov 05 06:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_0(8165464) WEXITSTATUS 0 Nov 05 06:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_0(8165464) done Nov 05 06:10:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5937(8162932) WEXITSTATUS 1 Nov 05 06:10:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5937(8162932) done Nov 05 06:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5932(8162927) WEXITSTATUS 1 Nov 05 06:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5932(8162927) done Nov 05 06:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_389(8163922) WEXITSTATUS 0 Nov 05 06:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_389(8163922) done Nov 05 06:11:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6596(8164638) WEXITSTATUS 1 Nov 05 06:11:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6596(8164638) done Nov 05 06:11:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_267(8163737) WEXITSTATUS 0 Nov 05 06:11:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_267(8163737) done Nov 05 06:11:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_1(8165465) WEXITSTATUS 0 Nov 05 06:11:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_1(8165465) done Nov 05 06:11:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3831(8165428) WEXITSTATUS 0 Nov 05 06:11:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3831(8165428) done Nov 05 06:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1786(8158049) WEXITSTATUS 0 Nov 05 06:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1786(8158049) done Nov 05 06:11:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2677(8153064) WEXITSTATUS 0 Nov 05 06:11:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2677(8153064) done Nov 05 06:11:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:11:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6712(8164823) WEXITSTATUS 1 Nov 05 06:11:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6712(8164823) done Nov 05 06:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6546(8164499) WEXITSTATUS 1 Nov 05 06:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6546(8164499) done Nov 05 06:11:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_2(8165480) NodeList=r909u07n02 #CPUs=16 Partition=day Nov 05 06:11:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_3(8165481) NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 06:11:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_4(8165482) NodeList=r815u35n02 #CPUs=16 Partition=day Nov 05 06:11:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_337(8114604) WEXITSTATUS 0 Nov 05 06:11:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_337(8114604) done Nov 05 06:11:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_66(8114332) WEXITSTATUS 0 Nov 05 06:11:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_66(8114332) done Nov 05 06:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1753(8157782) WEXITSTATUS 0 Nov 05 06:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1753(8157782) done Nov 05 06:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_447(8163990) WEXITSTATUS 0 Nov 05 06:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_447(8163990) done Nov 05 06:11:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_161(8163577) WEXITSTATUS 0 Nov 05 06:11:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_161(8163577) done Nov 05 06:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_326(8163827) WEXITSTATUS 0 Nov 05 06:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_326(8163827) done Nov 05 06:11:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:11:48 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:11:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:11:50 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:11:50 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:11:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2534(8127648) Nov 05 06:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_289(8163775) WEXITSTATUS 0 Nov 05 06:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_289(8163775) done Nov 05 06:12:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_365(8163886) WEXITSTATUS 0 Nov 05 06:12:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_365(8163886) done Nov 05 06:12:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_252(8163722) WEXITSTATUS 0 Nov 05 06:12:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_252(8163722) done Nov 05 06:12:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_279(8163765) WEXITSTATUS 0 Nov 05 06:12:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_279(8163765) done Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:12:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:12:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_298(8163784) WEXITSTATUS 0 Nov 05 06:12:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_298(8163784) done Nov 05 06:12:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_367(8163888) WEXITSTATUS 0 Nov 05 06:12:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_367(8163888) done Nov 05 06:12:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_269(8163739) WEXITSTATUS 0 Nov 05 06:12:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_269(8163739) done Nov 05 06:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_3(8165481) WEXITSTATUS 0 Nov 05 06:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_3(8165481) done Nov 05 06:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_2(8165480) WEXITSTATUS 0 Nov 05 06:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_2(8165480) done Nov 05 06:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_4(8165482) WEXITSTATUS 0 Nov 05 06:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_4(8165482) done Nov 05 06:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6207(8163343) WEXITSTATUS 1 Nov 05 06:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6207(8163343) done Nov 05 06:12:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_5(8165483) NodeList=r909u07n02 #CPUs=16 Partition=day Nov 05 06:12:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_6(8165484) NodeList=r815u35n02 #CPUs=16 Partition=day Nov 05 06:12:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_7(8165485) NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 06:12:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_8(8165486) NodeList=r815u15n01 #CPUs=16 Partition=day Nov 05 06:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_244(8114510) WEXITSTATUS 0 Nov 05 06:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_244(8114510) done Nov 05 06:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_365(8114632) WEXITSTATUS 0 Nov 05 06:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_365(8114632) done Nov 05 06:12:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_198(8163633) WEXITSTATUS 0 Nov 05 06:12:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_198(8163633) done Nov 05 06:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_16(8158177) WEXITSTATUS 1 Nov 05 06:12:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_16(8158177) done Nov 05 06:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7001(8165395) WEXITSTATUS 1 Nov 05 06:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7001(8165395) done Nov 05 06:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6689(8164791) WEXITSTATUS 1 Nov 05 06:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6689(8164791) done Nov 05 06:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_65(8135764) WEXITSTATUS 0 Nov 05 06:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_65(8135764) done Nov 05 06:12:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3834(8165487) in gpu on r808u17n01 Nov 05 06:12:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2350(8165488) in day on r909u01n04 Nov 05 06:12:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2351(8165489) in day on r909u13n02 Nov 05 06:12:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2352(8165490) in day on r918u05n02 Nov 05 06:12:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2353(8165491) in day on r918u36n02 Nov 05 06:12:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2354(8165492) in day on r815u09n07 Nov 05 06:12:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_236(8163688) WEXITSTATUS 0 Nov 05 06:12:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_236(8163688) done Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7046(8165493) in scavenge on r907u09n03 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7047(8165494) in scavenge on r915u32n01 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7048(8165495) in scavenge on r917u01n02 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7049(8165496) in scavenge on r904u07n04 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7050(8165497) in scavenge on r904u04n04 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7051(8165498) in scavenge on r912u36n02 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7052(8165499) in scavenge on r912u36n03 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7053(8165500) in scavenge on r918u32n03 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7054(8165501) in scavenge on r918u32n04 Nov 05 06:12:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7055(8165502) in scavenge on r815u15n01 Nov 05 06:12:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_945(7573114) in scavenge on r903u12n01 Nov 05 06:12:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_955(7573124) in scavenge on r807u22n01 Nov 05 06:12:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1034(7573203) in scavenge on r807u24n01 Nov 05 06:12:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_769(8115038) in scavenge on r913u32n01 Nov 05 06:12:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_865(8115134) in scavenge on r913u35n01 Nov 05 06:13:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7020(8165437) WEXITSTATUS 1 Nov 05 06:13:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7020(8165437) done Nov 05 06:13:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_282(8163768) WEXITSTATUS 0 Nov 05 06:13:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_282(8163768) done Nov 05 06:13:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4746(8157096) WEXITSTATUS 1 Nov 05 06:13:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4746(8157096) done Nov 05 06:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_341(8163862) WEXITSTATUS 0 Nov 05 06:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_341(8163862) done Nov 05 06:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_249(8163719) WEXITSTATUS 0 Nov 05 06:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_249(8163719) done Nov 05 06:13:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6631(8164696) WEXITSTATUS 1 Nov 05 06:13:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6631(8164696) done Nov 05 06:13:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1823(8158479) WEXITSTATUS 0 Nov 05 06:13:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1823(8158479) done Nov 05 06:13:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3171(8131733) WEXITSTATUS 1 Nov 05 06:13:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3171(8131733) done Nov 05 06:13:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5391(8159329) WEXITSTATUS 1 Nov 05 06:13:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5391(8159329) done Nov 05 06:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2656(8153043) WEXITSTATUS 0 Nov 05 06:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2656(8153043) done Nov 05 06:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_247(8163717) WEXITSTATUS 0 Nov 05 06:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_247(8163717) done Nov 05 06:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_7(8165485) WEXITSTATUS 0 Nov 05 06:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_7(8165485) done Nov 05 06:13:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_6(8165484) WEXITSTATUS 0 Nov 05 06:13:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_6(8165484) done Nov 05 06:13:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_445(8163988) WEXITSTATUS 0 Nov 05 06:13:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_445(8163988) done Nov 05 06:13:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_5(8165483) WEXITSTATUS 0 Nov 05 06:13:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_5(8165483) done Nov 05 06:13:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_9(8165503) NodeList=r909u07n02 #CPUs=16 Partition=day Nov 05 06:13:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_10(8165504) NodeList=r815u35n02 #CPUs=16 Partition=day Nov 05 06:13:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_11(8165505) NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 06:13:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_12(8165506) NodeList=r815u15n02 #CPUs=16 Partition=day Nov 05 06:13:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_28(8114294) WEXITSTATUS 0 Nov 05 06:13:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_28(8114294) done Nov 05 06:13:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_478(8114745) WEXITSTATUS 0 Nov 05 06:13:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_478(8114745) done Nov 05 06:13:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_8(8165486) WEXITSTATUS 0 Nov 05 06:13:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_8(8165486) done Nov 05 06:13:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_329(8163830) WEXITSTATUS 0 Nov 05 06:13:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_329(8163830) done Nov 05 06:14:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_414(8163947) WEXITSTATUS 0 Nov 05 06:14:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_414(8163947) done Nov 05 06:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_176(8163611) WEXITSTATUS 0 Nov 05 06:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_176(8163611) done Nov 05 06:14:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_371(8163892) WEXITSTATUS 0 Nov 05 06:14:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_371(8163892) done Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1771(8157939) WEXITSTATUS 0 Nov 05 06:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1771(8157939) done Nov 05 06:14:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_244(8163714) WEXITSTATUS 0 Nov 05 06:14:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_244(8163714) done Nov 05 06:14:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_327(8163828) WEXITSTATUS 0 Nov 05 06:14:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_327(8163828) done Nov 05 06:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_305(8163806) WEXITSTATUS 0 Nov 05 06:14:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_305(8163806) done Nov 05 06:14:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_353(8163874) WEXITSTATUS 0 Nov 05 06:14:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_353(8163874) done Nov 05 06:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_11(8165505) WEXITSTATUS 0 Nov 05 06:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_11(8165505) done Nov 05 06:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_10(8165504) WEXITSTATUS 0 Nov 05 06:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_10(8165504) done Nov 05 06:14:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_13(8165507) NodeList=r913u13n03 #CPUs=16 Partition=day Nov 05 06:14:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_14(8165508) NodeList=r815u35n02 #CPUs=16 Partition=day Nov 05 06:14:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_15(8165509) NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 06:14:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_16(8165510) NodeList=r815u15n01 #CPUs=16 Partition=day Nov 05 06:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_533(8114800) WEXITSTATUS 0 Nov 05 06:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_533(8114800) done Nov 05 06:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1130(7573299) WEXITSTATUS 0 Nov 05 06:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1130(7573299) done Nov 05 06:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_482(8164035) WEXITSTATUS 0 Nov 05 06:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_482(8164035) done Nov 05 06:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_9(8165503) WEXITSTATUS 0 Nov 05 06:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_9(8165503) done Nov 05 06:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_41(8135740) WEXITSTATUS 0 Nov 05 06:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_41(8135740) done Nov 05 06:14:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_12(8165506) WEXITSTATUS 0 Nov 05 06:14:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_12(8165506) done Nov 05 06:14:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_735(7572904) Nov 05 06:14:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2535(8127658) Nov 05 06:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_248(8163718) WEXITSTATUS 0 Nov 05 06:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_248(8163718) done Nov 05 06:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_26(8161233) WEXITSTATUS 0 Nov 05 06:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_26(8161233) done Nov 05 06:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6796(8164952) WEXITSTATUS 1 Nov 05 06:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6796(8164952) done Nov 05 06:15:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_187(8163622) WEXITSTATUS 0 Nov 05 06:15:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_187(8163622) done Nov 05 06:15:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_430(8163973) WEXITSTATUS 0 Nov 05 06:15:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_430(8163973) done Nov 05 06:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_258(8163728) WEXITSTATUS 0 Nov 05 06:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_258(8163728) done Nov 05 06:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_485(8164038) WEXITSTATUS 0 Nov 05 06:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_485(8164038) done Nov 05 06:15:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_356(8163877) WEXITSTATUS 0 Nov 05 06:15:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_356(8163877) done Nov 05 06:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6975(8165321) WEXITSTATUS 1 Nov 05 06:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6975(8165321) done Nov 05 06:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7028(8165453) WEXITSTATUS 1 Nov 05 06:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7028(8165453) done Nov 05 06:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_202(8163654) WEXITSTATUS 0 Nov 05 06:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_202(8163654) done Nov 05 06:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_15(8165509) WEXITSTATUS 0 Nov 05 06:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_15(8165509) done Nov 05 06:15:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_14(8165508) WEXITSTATUS 0 Nov 05 06:15:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_14(8165508) done Nov 05 06:15:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165463_17(8165511) in day on r807u05n04 Nov 05 06:15:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165463_18(8165512) in day on r909u07n02 Nov 05 06:15:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2355(8165513) in day on r918u05n01 Nov 05 06:15:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_13(8165507) WEXITSTATUS 0 Nov 05 06:15:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_13(8165507) done Nov 05 06:15:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_235(8163687) WEXITSTATUS 0 Nov 05 06:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_235(8163687) done Nov 05 06:15:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_19(8165514) NodeList=r913u13n03 #CPUs=16 Partition=day Nov 05 06:15:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_20(8165515) NodeList=r815u35n02 #CPUs=16 Partition=day Nov 05 06:15:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_21(8165516) NodeList=r816u15n04 #CPUs=16 Partition=day Nov 05 06:15:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_22(8165517) NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 06:15:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_23(8165518) NodeList=r815u15n02 #CPUs=16 Partition=day Nov 05 06:15:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_794(7572963) WEXITSTATUS 0 Nov 05 06:15:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_794(7572963) done Nov 05 06:15:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_16(8165510) WEXITSTATUS 0 Nov 05 06:15:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_16(8165510) done Nov 05 06:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_284(8163770) WEXITSTATUS 0 Nov 05 06:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_284(8163770) done Nov 05 06:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6845(8165045) WEXITSTATUS 1 Nov 05 06:15:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6845(8165045) done Nov 05 06:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_405(8163938) WEXITSTATUS 0 Nov 05 06:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_405(8163938) done Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7056(8165519) in scavenge on r904u01n03 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7057(8165520) in scavenge on r917u13n02 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7058(8165521) in scavenge on r815u29n08 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7059(8165522) in scavenge on r903u34n01 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7060(8165523) in scavenge on r912u36n03 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7061(8165524) in scavenge on r805u06n04 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7062(8165525) in scavenge on r805u11n03 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7063(8165526) in scavenge on r815u09n06 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7064(8165527) in scavenge on r815u35n03 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7065(8165528) in scavenge on r815u35n05 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_941(8115210) in scavenge on r816u35n11 Nov 05 06:15:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_992(8115261) in scavenge on r903u26n01 Nov 05 06:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_174(8163609) WEXITSTATUS 0 Nov 05 06:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_174(8163609) done Nov 05 06:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_330(8163831) WEXITSTATUS 0 Nov 05 06:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_330(8163831) done Nov 05 06:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1686(8156460) WEXITSTATUS 0 Nov 05 06:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1686(8156460) done Nov 05 06:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6797(8164953) WEXITSTATUS 1 Nov 05 06:15:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6797(8164953) done Nov 05 06:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_254(8163724) WEXITSTATUS 0 Nov 05 06:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_254(8163724) done Nov 05 06:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_212(8163664) WEXITSTATUS 0 Nov 05 06:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_212(8163664) done Nov 05 06:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1685(8156459) WEXITSTATUS 0 Nov 05 06:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1685(8156459) done Nov 05 06:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1830(8158509) WEXITSTATUS 0 Nov 05 06:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1830(8158509) done Nov 05 06:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_288(8163774) WEXITSTATUS 0 Nov 05 06:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_288(8163774) done Nov 05 06:15:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_259(8163729) WEXITSTATUS 0 Nov 05 06:15:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_259(8163729) done Nov 05 06:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1836(8158615) WEXITSTATUS 0 Nov 05 06:16:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1836(8158615) done Nov 05 06:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_325(8163826) WEXITSTATUS 0 Nov 05 06:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_325(8163826) done Nov 05 06:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_468(8164021) WEXITSTATUS 0 Nov 05 06:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_468(8164021) done Nov 05 06:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1861(8160522) WEXITSTATUS 0 Nov 05 06:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1861(8160522) done Nov 05 06:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_283(8163769) WEXITSTATUS 0 Nov 05 06:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_283(8163769) done Nov 05 06:16:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7047(8165494) WEXITSTATUS 1 Nov 05 06:16:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7047(8165494) done Nov 05 06:16:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_266(8163736) WEXITSTATUS 0 Nov 05 06:16:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_266(8163736) done Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6838(8165033) WEXITSTATUS 1 Nov 05 06:16:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6838(8165033) done Nov 05 06:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6624(8164689) WEXITSTATUS 1 Nov 05 06:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6624(8164689) done Nov 05 06:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_400(8163933) WEXITSTATUS 0 Nov 05 06:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_400(8163933) done Nov 05 06:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1664(8156297) WEXITSTATUS 0 Nov 05 06:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1664(8156297) done Nov 05 06:16:22 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6108(8163146) has been requeued to reclaim resources for JobId=Invalid Nov 05 06:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_9(8158170) WEXITSTATUS 1 Nov 05 06:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_9(8158170) done Nov 05 06:16:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6108(8163146) Nov 05 06:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3833(8165466) WEXITSTATUS 0 Nov 05 06:16:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3833(8165466) done Nov 05 06:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_68(8135767) WEXITSTATUS 0 Nov 05 06:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_68(8135767) done Nov 05 06:16:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_296(8163782) WEXITSTATUS 0 Nov 05 06:16:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_296(8163782) done Nov 05 06:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_22(8165517) WEXITSTATUS 0 Nov 05 06:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_22(8165517) done Nov 05 06:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_18(8165512) WEXITSTATUS 0 Nov 05 06:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_18(8165512) done Nov 05 06:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_20(8165515) WEXITSTATUS 0 Nov 05 06:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_20(8165515) done Nov 05 06:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_17(8165511) WEXITSTATUS 0 Nov 05 06:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_17(8165511) done Nov 05 06:16:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_21(8165516) WEXITSTATUS 0 Nov 05 06:16:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_21(8165516) done Nov 05 06:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_19(8165514) WEXITSTATUS 0 Nov 05 06:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_19(8165514) done Nov 05 06:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_23(8165518) WEXITSTATUS 0 Nov 05 06:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_23(8165518) done Nov 05 06:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_403(8163936) WEXITSTATUS 0 Nov 05 06:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_403(8163936) done Nov 05 06:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2649(8153036) WEXITSTATUS 0 Nov 05 06:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2649(8153036) done Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_24(8165529) NodeList=r805u27n02 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_25(8165530) NodeList=r807u05n04 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_26(8165531) NodeList=r909u07n02 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_27(8165532) NodeList=r913u13n03 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_28(8165533) NodeList=r918u32n01 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_29(8165534) NodeList=r815u35n02 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_30(8165535) NodeList=r816u15n04 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_31(8165536) NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_32(8165537) NodeList=r815u15n01 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_33(8165538) NodeList=r815u15n02 #CPUs=16 Partition=day Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_853(7571819) WEXITSTATUS 0 Nov 05 06:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_853(7571819) done Nov 05 06:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_243(8163713) WEXITSTATUS 0 Nov 05 06:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_243(8163713) done Nov 05 06:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_147(8163563) WEXITSTATUS 0 Nov 05 06:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_147(8163563) done Nov 05 06:16:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_343(8163864) WEXITSTATUS 0 Nov 05 06:16:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_343(8163864) done Nov 05 06:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6643(8164720) WEXITSTATUS 1 Nov 05 06:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6643(8164720) done Nov 05 06:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_407(8163940) WEXITSTATUS 0 Nov 05 06:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_407(8163940) done Nov 05 06:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1667(8156300) WEXITSTATUS 0 Nov 05 06:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1667(8156300) done Nov 05 06:16:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:16:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_432(8163975) WEXITSTATUS 0 Nov 05 06:16:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_432(8163975) done Nov 05 06:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_229(8163681) WEXITSTATUS 0 Nov 05 06:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_229(8163681) done Nov 05 06:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5368(8159178) WEXITSTATUS 1 Nov 05 06:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5368(8159178) done Nov 05 06:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_237(8163689) WEXITSTATUS 0 Nov 05 06:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_237(8163689) done Nov 05 06:17:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1863(8160695) WEXITSTATUS 0 Nov 05 06:17:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1863(8160695) done Nov 05 06:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6888(8165127) WEXITSTATUS 1 Nov 05 06:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6888(8165127) done Nov 05 06:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1849(8159027) WEXITSTATUS 0 Nov 05 06:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1849(8159027) done Nov 05 06:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1792(8158103) WEXITSTATUS 0 Nov 05 06:17:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1792(8158103) done Nov 05 06:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1733(8157683) WEXITSTATUS 0 Nov 05 06:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1733(8157683) done Nov 05 06:17:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_325(7571290) Nov 05 06:17:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_834(7573003) Nov 05 06:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1816(8158428) WEXITSTATUS 0 Nov 05 06:17:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1816(8158428) done Nov 05 06:17:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_24(8165529) WEXITSTATUS 0 Nov 05 06:17:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_24(8165529) done Nov 05 06:17:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_31(8165536) WEXITSTATUS 0 Nov 05 06:17:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_31(8165536) done Nov 05 06:17:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 06:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_446(8163989) WEXITSTATUS 0 Nov 05 06:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_446(8163989) done Nov 05 06:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_29(8165534) WEXITSTATUS 0 Nov 05 06:17:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_29(8165534) done Nov 05 06:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_26(8165531) WEXITSTATUS 0 Nov 05 06:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_26(8165531) done Nov 05 06:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_27(8165532) WEXITSTATUS 0 Nov 05 06:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_27(8165532) done Nov 05 06:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_25(8165530) WEXITSTATUS 0 Nov 05 06:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_25(8165530) done Nov 05 06:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_30(8165535) WEXITSTATUS 0 Nov 05 06:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_30(8165535) done Nov 05 06:17:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_28(8165533) WEXITSTATUS 0 Nov 05 06:17:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_28(8165533) done Nov 05 06:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_33(8165538) WEXITSTATUS 0 Nov 05 06:17:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_33(8165538) done Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_34(8165539) NodeList=r805u27n02 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_35(8165540) NodeList=r807u05n04 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_36(8165541) NodeList=r909u07n02 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_37(8165542) NodeList=r913u13n03 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_38(8165543) NodeList=r918u32n01 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_39(8165544) NodeList=r919u32n01 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_40(8165545) NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_41(8165546) NodeList=r816u15n04 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_42(8165547) NodeList=r815u23n05 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_43(8165548) NodeList=r815u35n02 #CPUs=16 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2356(8165549) NodeList=r806u06n02 #CPUs=1 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2357(8165550) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2358(8165551) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2359(8165552) NodeList=r918u09n02 #CPUs=1 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2360(8165553) NodeList=r919u01n01 #CPUs=1 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2361(8165554) NodeList=r919u09n01 #CPUs=1 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2362(8165555) NodeList=r816u03n05 #CPUs=1 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2363(8165556) NodeList=r815u23n05 #CPUs=1 Partition=day Nov 05 06:17:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2364(8165557) NodeList=r815u29n01 #CPUs=1 Partition=day Nov 05 06:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_32(8165537) WEXITSTATUS 0 Nov 05 06:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_32(8165537) done Nov 05 06:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_239(8163691) WEXITSTATUS 0 Nov 05 06:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_239(8163691) done Nov 05 06:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2665(8153052) WEXITSTATUS 0 Nov 05 06:17:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2665(8153052) done Nov 05 06:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6043(8163065) WEXITSTATUS 1 Nov 05 06:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6043(8163065) done Nov 05 06:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_483(8164036) WEXITSTATUS 0 Nov 05 06:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_483(8164036) done Nov 05 06:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_246(8163716) WEXITSTATUS 0 Nov 05 06:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_246(8163716) done Nov 05 06:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_391(8163924) WEXITSTATUS 0 Nov 05 06:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_391(8163924) done Nov 05 06:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_281(8163767) WEXITSTATUS 0 Nov 05 06:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_281(8163767) done Nov 05 06:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_461(8164014) WEXITSTATUS 0 Nov 05 06:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_461(8164014) done Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:18:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:18:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3835(8165558) in gpu on r808u08n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7066(8165559) in scavenge on r801u14n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7067(8165560) in scavenge on r801u14n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7068(8165561) in scavenge on r802u02n03 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7069(8165562) in scavenge on r802u09n02 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7070(8165563) in scavenge on r915u32n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7071(8165564) in scavenge on r917u13n04 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7072(8165565) in scavenge on r917u29n04 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7073(8165566) in scavenge on r816u23n12 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7074(8165567) in scavenge on r903u24n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165463_44(8165568) in scavenge on r912u05n03 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165463_45(8165569) in scavenge on r816u03n12 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165463_46(8165570) in scavenge on r816u23n09 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8165463_47(8165571) in scavenge on r815u15n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1016(8115285) in scavenge on r907u28n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1101(8115370) in scavenge on r805u02n02 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1115(8115384) in scavenge on r913u01n04 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1197(8115467) in scavenge on r904u10n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_488(7572656) in scavenge on r918u32n01 Nov 05 06:18:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_844(7573013) in scavenge on r919u32n01 Nov 05 06:18:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_280(8163766) WEXITSTATUS 0 Nov 05 06:18:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_280(8163766) done Nov 05 06:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_34(8165539) WEXITSTATUS 0 Nov 05 06:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_34(8165539) done Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2365(8165572) in scavenge on r911u05n01 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2366(8165573) in scavenge on r801u09n03 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2367(8165574) in scavenge on r801u09n04 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2368(8165575) in scavenge on r917u13n02 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2369(8165576) in scavenge on r917u13n04 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2370(8165577) in scavenge on r815u03n12 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2371(8165578) in scavenge on r816u23n08 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2372(8165579) in scavenge on r815u23n07 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2373(8165580) in scavenge on r815u23n07 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2374(8165581) in scavenge on r815u29n08 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2375(8165582) in scavenge on r912u09n01 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2376(8165583) in scavenge on r903u24n01 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2377(8165584) in scavenge on r903u26n01 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2378(8165585) in scavenge on r816u35n06 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2379(8165586) in scavenge on r816u35n06 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2380(8165587) in scavenge on r806u31n01 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2381(8165588) in scavenge on r806u31n01 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2382(8165589) in scavenge on r805u34n01 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_40(8165545) WEXITSTATUS 0 Nov 05 06:18:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_40(8165545) done Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_43(8165548) WEXITSTATUS 0 Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_43(8165548) done Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_41(8165546) WEXITSTATUS 0 Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_41(8165546) done Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_42(8165547) WEXITSTATUS 0 Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_42(8165547) done Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_38(8165543) WEXITSTATUS 0 Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_38(8165543) done Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_180(8163615) WEXITSTATUS 0 Nov 05 06:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_180(8163615) done Nov 05 06:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_35(8165540) WEXITSTATUS 0 Nov 05 06:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_35(8165540) done Nov 05 06:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_37(8165542) WEXITSTATUS 0 Nov 05 06:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_37(8165542) done Nov 05 06:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_36(8165541) WEXITSTATUS 0 Nov 05 06:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_36(8165541) done Nov 05 06:18:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_48(8165590) NodeList=r805u27n02 #CPUs=16 Partition=day Nov 05 06:18:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165463_49(8165463) NodeList=r807u05n04 #CPUs=16 Partition=day Nov 05 06:18:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_488(7572656) WEXITSTATUS 0 Nov 05 06:18:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_488(7572656) done Nov 05 06:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_411(8163944) WEXITSTATUS 0 Nov 05 06:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_411(8163944) done Nov 05 06:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_39(8165544) WEXITSTATUS 0 Nov 05 06:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_39(8165544) done Nov 05 06:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_320(8163821) WEXITSTATUS 0 Nov 05 06:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_320(8163821) done Nov 05 06:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_242(8163712) WEXITSTATUS 0 Nov 05 06:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_242(8163712) done Nov 05 06:18:48 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8165591 NodeList=(null) usec=28109 Nov 05 06:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6646(8164723) WEXITSTATUS 1 Nov 05 06:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6646(8164723) done Nov 05 06:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_338(8163859) WEXITSTATUS 0 Nov 05 06:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_338(8163859) done Nov 05 06:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_181(8163616) WEXITSTATUS 0 Nov 05 06:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_181(8163616) done Nov 05 06:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1585(8151845) WEXITSTATUS 0 Nov 05 06:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1585(8151845) done Nov 05 06:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_363(8163884) WEXITSTATUS 0 Nov 05 06:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_363(8163884) done Nov 05 06:19:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1041(8128645) WEXITSTATUS 0 Nov 05 06:19:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1041(8128645) done Nov 05 06:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_277(8163763) WEXITSTATUS 0 Nov 05 06:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_277(8163763) done Nov 05 06:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1748(8157765) WEXITSTATUS 0 Nov 05 06:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1748(8157765) done Nov 05 06:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_474(8164027) WEXITSTATUS 0 Nov 05 06:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_474(8164027) done Nov 05 06:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_349(8163870) WEXITSTATUS 0 Nov 05 06:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_349(8163870) done Nov 05 06:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_342(8163863) WEXITSTATUS 0 Nov 05 06:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_342(8163863) done Nov 05 06:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_307(8163808) WEXITSTATUS 0 Nov 05 06:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_307(8163808) done Nov 05 06:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_409(8163942) WEXITSTATUS 0 Nov 05 06:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_409(8163942) done Nov 05 06:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_458(8164011) WEXITSTATUS 0 Nov 05 06:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_458(8164011) done Nov 05 06:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_45(8165569) WEXITSTATUS 0 Nov 05 06:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_45(8165569) done Nov 05 06:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_46(8165570) WEXITSTATUS 0 Nov 05 06:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_46(8165570) done Nov 05 06:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_44(8165568) WEXITSTATUS 0 Nov 05 06:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_44(8165568) done Nov 05 06:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_47(8165571) WEXITSTATUS 0 Nov 05 06:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_47(8165571) done Nov 05 06:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_285(8163771) WEXITSTATUS 0 Nov 05 06:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_285(8163771) done Nov 05 06:19:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_48(8165590) WEXITSTATUS 0 Nov 05 06:19:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_48(8165590) done Nov 05 06:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1732(8157682) WEXITSTATUS 0 Nov 05 06:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1732(8157682) done Nov 05 06:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_406(8163939) WEXITSTATUS 0 Nov 05 06:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_406(8163939) done Nov 05 06:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_49(8165463) WEXITSTATUS 0 Nov 05 06:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165463_49(8165463) done Nov 05 06:19:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165591 NodeList=r903u37n01 #CPUs=10 Partition=gpu_devel Nov 05 06:19:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2383(8165592) NodeList=r909u29n03 #CPUs=1 Partition=day Nov 05 06:19:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2384(8165593) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 06:19:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2385(8165594) NodeList=r918u32n01 #CPUs=1 Partition=day Nov 05 06:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_604(7571569) WEXITSTATUS 0 Nov 05 06:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_604(7571569) done Nov 05 06:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_308(8163809) WEXITSTATUS 0 Nov 05 06:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_308(8163809) done Nov 05 06:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1760(8157829) WEXITSTATUS 0 Nov 05 06:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1760(8157829) done Nov 05 06:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_442(8163985) WEXITSTATUS 0 Nov 05 06:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_442(8163985) done Nov 05 06:19:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2540(8127672) Nov 05 06:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_462(8164015) WEXITSTATUS 0 Nov 05 06:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_462(8164015) done Nov 05 06:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_421(8163964) WEXITSTATUS 0 Nov 05 06:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_421(8163964) done Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5286(8158734) WEXITSTATUS 1 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5286(8158734) done Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_402(8163935) WEXITSTATUS 0 Nov 05 06:20:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_402(8163935) done Nov 05 06:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_250(8163720) WEXITSTATUS 0 Nov 05 06:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_250(8163720) done Nov 05 06:20:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2386(8165595) NodeList=r918u36n04 #CPUs=1 Partition=day Nov 05 06:20:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2387(8165596) NodeList=r919u32n01 #CPUs=1 Partition=day Nov 05 06:20:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144141 NodeList=r807u39n01,r912u05n01 #CPUs=68 Partition=scavenge Nov 05 06:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_271(8163757) WEXITSTATUS 0 Nov 05 06:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_271(8163757) done Nov 05 06:20:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_209(8163661) WEXITSTATUS 0 Nov 05 06:20:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_209(8163661) done Nov 05 06:20:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_337(8163858) WEXITSTATUS 0 Nov 05 06:20:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_337(8163858) done Nov 05 06:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_255(8163725) WEXITSTATUS 0 Nov 05 06:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_255(8163725) done Nov 05 06:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_520(8164085) WEXITSTATUS 0 Nov 05 06:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_520(8164085) done Nov 05 06:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_436(8163979) WEXITSTATUS 0 Nov 05 06:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_436(8163979) done Nov 05 06:21:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_245(8163715) WEXITSTATUS 0 Nov 05 06:21:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_245(8163715) done Nov 05 06:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_276(8163762) WEXITSTATUS 0 Nov 05 06:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_276(8163762) done Nov 05 06:21:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7075(8165597) in scavenge on r917u13n02 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7076(8165598) in scavenge on r805u27n02 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7077(8165599) in scavenge on r805u27n02 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_549(7572718) in scavenge on r917u13n03 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1125(7570787) in scavenge on r917u29n01 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_271(7571236) in scavenge on r807u05n04 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_569(8114836) in scavenge on r909u07n02 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1055(7572021) in scavenge on r909u07n02 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_785(7571750) in scavenge on r913u13n03 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_139(8115609) in scavenge on r913u13n03 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_158(8115628) in scavenge on r815u15n01 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_283(7572451) in scavenge on r816u15n04 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_946(7573115) in scavenge on r816u15n04 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_540(7571505) in scavenge on r815u23n05 Nov 05 06:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_668(7571633) in scavenge on r815u35n02 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2388(8165600) in scavenge on r915u35n02 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2389(8165601) in scavenge on r915u35n02 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2390(8165602) in scavenge on r917u13n04 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2391(8165603) in scavenge on r917u29n04 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2392(8165604) in scavenge on r815u09n10 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2393(8165605) in scavenge on r807u24n01 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2394(8165606) in scavenge on r806u28n01 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2395(8165607) in scavenge on r806u28n01 Nov 05 06:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2396(8165608) in scavenge on r915u09n01 Nov 05 06:21:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_375(8163908) WEXITSTATUS 0 Nov 05 06:21:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_375(8163908) done Nov 05 06:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_465(8164018) WEXITSTATUS 0 Nov 05 06:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_465(8164018) done Nov 05 06:21:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3783(8164718) WEXITSTATUS 0 Nov 05 06:21:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3783(8164718) done Nov 05 06:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6765(8164914) WEXITSTATUS 1 Nov 05 06:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6765(8164914) done Nov 05 06:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7044(8165478) WEXITSTATUS 1 Nov 05 06:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7044(8165478) done Nov 05 06:21:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_340(8163861) WEXITSTATUS 0 Nov 05 06:21:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_340(8163861) done Nov 05 06:21:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2400(8152444) WEXITSTATUS 0 Nov 05 06:21:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2400(8152444) done Nov 05 06:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_241(8163711) WEXITSTATUS 0 Nov 05 06:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_241(8163711) done Nov 05 06:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_444(8163987) WEXITSTATUS 0 Nov 05 06:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_444(8163987) done Nov 05 06:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1504(8142945) WEXITSTATUS 0 Nov 05 06:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1504(8142945) done Nov 05 06:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_399(8163932) WEXITSTATUS 0 Nov 05 06:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_399(8163932) done Nov 05 06:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6777(8164929) WEXITSTATUS 1 Nov 05 06:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6777(8164929) done Nov 05 06:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6615(8164675) WEXITSTATUS 1 Nov 05 06:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6615(8164675) done Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_357(8163878) WEXITSTATUS 0 Nov 05 06:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_357(8163878) done Nov 05 06:22:23 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2543(8127680) Nov 05 06:22:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_257(8163727) WEXITSTATUS 0 Nov 05 06:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_257(8163727) done Nov 05 06:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_392(8163925) WEXITSTATUS 0 Nov 05 06:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_392(8163925) done Nov 05 06:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_394(8163927) WEXITSTATUS 0 Nov 05 06:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_394(8163927) done Nov 05 06:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7055(8165502) WEXITSTATUS 1 Nov 05 06:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7055(8165502) done Nov 05 06:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_352(8163873) WEXITSTATUS 0 Nov 05 06:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_352(8163873) done Nov 05 06:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1705(8156850) WEXITSTATUS 0 Nov 05 06:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1705(8156850) done Nov 05 06:22:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2397(8165609) NodeList=r919u01n02 #CPUs=1 Partition=day Nov 05 06:22:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_351(8163872) WEXITSTATUS 0 Nov 05 06:22:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_351(8163872) done Nov 05 06:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_429(8163972) WEXITSTATUS 0 Nov 05 06:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_429(8163972) done Nov 05 06:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_413(8163946) WEXITSTATUS 0 Nov 05 06:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_413(8163946) done Nov 05 06:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_514(8164079) WEXITSTATUS 0 Nov 05 06:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_514(8164079) done Nov 05 06:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6574(8164611) WEXITSTATUS 1 Nov 05 06:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6574(8164611) done Nov 05 06:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1703(8156848) WEXITSTATUS 0 Nov 05 06:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1703(8156848) done Nov 05 06:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_484(8164037) WEXITSTATUS 0 Nov 05 06:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_484(8164037) done Nov 05 06:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6519(8164454) WEXITSTATUS 1 Nov 05 06:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6519(8164454) done Nov 05 06:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_321(8163822) WEXITSTATUS 0 Nov 05 06:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_321(8163822) done Nov 05 06:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_428(8163971) WEXITSTATUS 0 Nov 05 06:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_428(8163971) done Nov 05 06:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_448(8163991) WEXITSTATUS 0 Nov 05 06:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_448(8163991) done Nov 05 06:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2398(8165610) NodeList=r918u36n01 #CPUs=1 Partition=day Nov 05 06:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_362(8163883) WEXITSTATUS 0 Nov 05 06:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_362(8163883) done Nov 05 06:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_302(8163803) WEXITSTATUS 0 Nov 05 06:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_302(8163803) done Nov 05 06:23:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3836(8165611) in gpu on r903u34n01 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7078(8165612) in scavenge on r917u07n04 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7079(8165613) in scavenge on r917u29n03 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7080(8165614) in scavenge on r815u09n12 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7081(8165615) in scavenge on r909u13n03 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7082(8165616) in scavenge on r807u05n04 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7083(8165617) in scavenge on r807u08n01 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7084(8165618) in scavenge on r807u08n01 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7085(8165619) in scavenge on r918u32n03 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_396(7571361) in scavenge on r801u32n02 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1157(7572124) in scavenge on r917u29n02 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_87(7572254) in scavenge on r917u29n04 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_163(7572331) in scavenge on r913u01n02 Nov 05 06:23:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_539(7572708) in scavenge on r815u15n01 Nov 05 06:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_311(8163812) WEXITSTATUS 0 Nov 05 06:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_311(8163812) done Nov 05 06:23:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6621(8164681) WEXITSTATUS 1 Nov 05 06:23:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6621(8164681) done Nov 05 06:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_344(8163865) WEXITSTATUS 0 Nov 05 06:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_344(8163865) done Nov 05 06:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_544(8164125) WEXITSTATUS 0 Nov 05 06:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_544(8164125) done Nov 05 06:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3836(8165611) WEXITSTATUS 0 Nov 05 06:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3836(8165611) done Nov 05 06:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6842(8165042) WEXITSTATUS 1 Nov 05 06:23:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6842(8165042) done Nov 05 06:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_208(8163660) WEXITSTATUS 0 Nov 05 06:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_208(8163660) done Nov 05 06:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6434(8164264) WEXITSTATUS 1 Nov 05 06:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6434(8164264) done Nov 05 06:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1742(8157745) WEXITSTATUS 0 Nov 05 06:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1742(8157745) done Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_304(8163805) WEXITSTATUS 0 Nov 05 06:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_304(8163805) done Nov 05 06:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1935(8125300) WEXITSTATUS 1 Nov 05 06:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1935(8125300) done Nov 05 06:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_500(8164065) WEXITSTATUS 0 Nov 05 06:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_500(8164065) done Nov 05 06:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1763(8157845) WEXITSTATUS 0 Nov 05 06:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1763(8157845) done Nov 05 06:24:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2399(8165620) NodeList=r918u09n03 #CPUs=1 Partition=day Nov 05 06:24:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2400(8165621) NodeList=r919u36n03 #CPUs=1 Partition=day Nov 05 06:24:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_206(8163658) WEXITSTATUS 0 Nov 05 06:24:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_206(8163658) done Nov 05 06:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_303(8163804) WEXITSTATUS 0 Nov 05 06:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_303(8163804) done Nov 05 06:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7073(8165566) WEXITSTATUS 1 Nov 05 06:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7073(8165566) done Nov 05 06:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6830(8165017) WEXITSTATUS 1 Nov 05 06:24:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6830(8165017) done Nov 05 06:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1806(8158219) WEXITSTATUS 0 Nov 05 06:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1806(8158219) done Nov 05 06:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_205(8163657) WEXITSTATUS 0 Nov 05 06:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_205(8163657) done Nov 05 06:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_309(8163810) WEXITSTATUS 0 Nov 05 06:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_309(8163810) done Nov 05 06:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_315(8163816) WEXITSTATUS 0 Nov 05 06:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_315(8163816) done Nov 05 06:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1693(8156715) WEXITSTATUS 0 Nov 05 06:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1693(8156715) done Nov 05 06:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_306(8163807) WEXITSTATUS 0 Nov 05 06:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_306(8163807) done Nov 05 06:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_385(8163918) WEXITSTATUS 0 Nov 05 06:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_385(8163918) done Nov 05 06:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1857(8159884) WEXITSTATUS 0 Nov 05 06:25:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1857(8159884) done Nov 05 06:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1842(8158627) WEXITSTATUS 0 Nov 05 06:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1842(8158627) done Nov 05 06:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1822(8158460) WEXITSTATUS 0 Nov 05 06:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1822(8158460) done Nov 05 06:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_360(8163881) WEXITSTATUS 0 Nov 05 06:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_360(8163881) done Nov 05 06:25:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_503(8164068) WEXITSTATUS 0 Nov 05 06:25:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_503(8164068) done Nov 05 06:25:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_339(8163860) WEXITSTATUS 0 Nov 05 06:25:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_339(8163860) done Nov 05 06:25:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_388(8163921) WEXITSTATUS 0 Nov 05 06:25:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_388(8163921) done Nov 05 06:25:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2401(8165622) NodeList=r919u05n01 #CPUs=1 Partition=day Nov 05 06:25:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2402(8165623) NodeList=r914u01n01 #CPUs=1 Partition=day Nov 05 06:25:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2403(8165624) NodeList=r914u01n04 #CPUs=1 Partition=day Nov 05 06:25:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2404(8165625) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 06:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_312(8163813) WEXITSTATUS 0 Nov 05 06:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_312(8163813) done Nov 05 06:25:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_274(8163760) WEXITSTATUS 0 Nov 05 06:25:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_274(8163760) done Nov 05 06:25:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_494(8164059) WEXITSTATUS 0 Nov 05 06:25:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_494(8164059) done Nov 05 06:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_419(8163962) WEXITSTATUS 0 Nov 05 06:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_419(8163962) done Nov 05 06:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_404(8163937) WEXITSTATUS 0 Nov 05 06:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_404(8163937) done Nov 05 06:25:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_417(8163960) WEXITSTATUS 0 Nov 05 06:25:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_417(8163960) done Nov 05 06:25:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6764(8164913) WEXITSTATUS 1 Nov 05 06:25:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6764(8164913) done Nov 05 06:26:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_383(8163916) WEXITSTATUS 0 Nov 05 06:26:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_383(8163916) done Nov 05 06:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6775(8164927) WEXITSTATUS 1 Nov 05 06:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6775(8164927) done Nov 05 06:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1870(8161560) WEXITSTATUS 0 Nov 05 06:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1870(8161560) done Nov 05 06:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_319(8163820) WEXITSTATUS 0 Nov 05 06:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_319(8163820) done Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:26:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6583(8164623) WEXITSTATUS 1 Nov 05 06:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6583(8164623) done Nov 05 06:26:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3837(8165626) in gpu on r903u34n01 Nov 05 06:26:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162653 WEXITSTATUS 0 Nov 05 06:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162653 done Nov 05 06:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_355(8163876) WEXITSTATUS 0 Nov 05 06:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_355(8163876) done Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7086(8165627) in scavenge on r907u01n04 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7087(8165628) in scavenge on r801u09n03 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7088(8165629) in scavenge on r912u05n04 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7089(8165630) in scavenge on r917u13n01 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7090(8165631) in scavenge on r802u23n02 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7091(8165632) in scavenge on r802u30n01 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7092(8165633) in scavenge on r816u23n12 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7093(8165634) in scavenge on r805u11n01 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7094(8165635) in scavenge on r919u32n03 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1091(7572057) in scavenge on r917u13n02 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1108(7572074) in scavenge on r917u13n03 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_423(7572591) in scavenge on r912u36n01 Nov 05 06:26:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_451(7572619) in scavenge on r913u28n02 Nov 05 06:26:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2405(8165636) in scavenge on r917u13n01 Nov 05 06:26:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2406(8165637) in scavenge on r917u29n03 Nov 05 06:26:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2407(8165638) in scavenge on r816u23n11 Nov 05 06:26:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2408(8165639) in scavenge on r912u09n01 Nov 05 06:26:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2409(8165640) in scavenge on r806u28n01 Nov 05 06:26:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2410(8165641) NodeList=r815u15n02 #CPUs=1 Partition=day Nov 05 06:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6770(8164919) WEXITSTATUS 1 Nov 05 06:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6770(8164919) done Nov 05 06:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_393(8163926) WEXITSTATUS 0 Nov 05 06:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_393(8163926) done Nov 05 06:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6410(8164183) WEXITSTATUS 1 Nov 05 06:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6410(8164183) done Nov 05 06:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6876(8165106) WEXITSTATUS 1 Nov 05 06:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6876(8165106) done Nov 05 06:26:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1746(8157763) WEXITSTATUS 0 Nov 05 06:26:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1746(8157763) done Nov 05 06:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_456(8164009) WEXITSTATUS 0 Nov 05 06:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_456(8164009) done Nov 05 06:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_1(8158200) WEXITSTATUS 1 Nov 05 06:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_1(8158200) done Nov 05 06:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162671 WEXITSTATUS 0 Nov 05 06:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162671 done Nov 05 06:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1656(8156241) WEXITSTATUS 0 Nov 05 06:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1656(8156241) done Nov 05 06:26:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_420(8163963) WEXITSTATUS 0 Nov 05 06:26:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_420(8163963) done Nov 05 06:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1717(8157211) WEXITSTATUS 0 Nov 05 06:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1717(8157211) done Nov 05 06:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6864(8165077) WEXITSTATUS 1 Nov 05 06:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6864(8165077) done Nov 05 06:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6566(8164592) WEXITSTATUS 1 Nov 05 06:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6566(8164592) done Nov 05 06:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_318(8163819) WEXITSTATUS 0 Nov 05 06:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_318(8163819) done Nov 05 06:27:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2411(8165642) NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 06:27:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2412(8165643) NodeList=r909u10n01 #CPUs=1 Partition=day Nov 05 06:27:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2413(8165644) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 06:27:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2414(8165645) NodeList=r914u05n01 #CPUs=1 Partition=day Nov 05 06:27:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2415(8165646) NodeList=r914u05n03 #CPUs=1 Partition=day Nov 05 06:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_273(8163759) WEXITSTATUS 0 Nov 05 06:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_273(8163759) done Nov 05 06:27:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8130506 WEXITSTATUS 0 Nov 05 06:27:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8130506 done Nov 05 06:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162674 WEXITSTATUS 0 Nov 05 06:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162674 done Nov 05 06:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_377(8163910) WEXITSTATUS 0 Nov 05 06:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_377(8163910) done Nov 05 06:27:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_463(8164016) WEXITSTATUS 0 Nov 05 06:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_463(8164016) done Nov 05 06:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_455(8164008) WEXITSTATUS 0 Nov 05 06:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_455(8164008) done Nov 05 06:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_278(8163764) WEXITSTATUS 0 Nov 05 06:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_278(8163764) done Nov 05 06:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_470(8164023) WEXITSTATUS 0 Nov 05 06:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_470(8164023) done Nov 05 06:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7018(8165435) WEXITSTATUS 1 Nov 05 06:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7018(8165435) done Nov 05 06:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_210(8163662) WEXITSTATUS 0 Nov 05 06:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_210(8163662) done Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:28:17 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_313(8163814) WEXITSTATUS 0 Nov 05 06:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_313(8163814) done Nov 05 06:28:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_346(8163867) WEXITSTATUS 0 Nov 05 06:28:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_346(8163867) done Nov 05 06:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1704(8156849) WEXITSTATUS 0 Nov 05 06:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1704(8156849) done Nov 05 06:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1735(8157685) WEXITSTATUS 0 Nov 05 06:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1735(8157685) done Nov 05 06:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_323(8163824) WEXITSTATUS 0 Nov 05 06:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_323(8163824) done Nov 05 06:28:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2416(8165647) NodeList=r807u11n01 #CPUs=1 Partition=day Nov 05 06:28:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2417(8165648) NodeList=r918u36n04 #CPUs=1 Partition=day Nov 05 06:28:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2418(8165649) NodeList=r919u05n03 #CPUs=1 Partition=day Nov 05 06:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6788(8164942) WEXITSTATUS 1 Nov 05 06:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6788(8164942) done Nov 05 06:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1802(8158198) WEXITSTATUS 0 Nov 05 06:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1802(8158198) done Nov 05 06:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_412(8163945) WEXITSTATUS 0 Nov 05 06:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_412(8163945) done Nov 05 06:28:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_433(8163976) WEXITSTATUS 0 Nov 05 06:28:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_433(8163976) done Nov 05 06:28:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3837(8165626) WEXITSTATUS 0 Nov 05 06:28:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3837(8165626) done Nov 05 06:29:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3838(8165650) in gpu on r903u34n01 Nov 05 06:29:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6606(8164661) WEXITSTATUS 1 Nov 05 06:29:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6606(8164661) done Nov 05 06:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_496(8164061) WEXITSTATUS 0 Nov 05 06:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_496(8164061) done Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7095(8165651) in scavenge on r802u06n04 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7096(8165652) in scavenge on r815u29n12 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7097(8165653) in scavenge on r807u11n01 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7098(8165654) in scavenge on r909u10n01 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7099(8165655) in scavenge on r914u05n03 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7100(8165656) in scavenge on r914u05n03 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7101(8165657) in scavenge on r914u05n03 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7102(8165658) in scavenge on r918u05n03 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_153(7572321) in scavenge on r917u13n01 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_666(7572835) in scavenge on r917u29n01 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_778(7572947) in scavenge on r805u25n04 Nov 05 06:29:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_975(7573144) in scavenge on r806u09n02 Nov 05 06:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_539(8164120) WEXITSTATUS 0 Nov 05 06:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_539(8164120) done Nov 05 06:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7046(8165493) WEXITSTATUS 1 Nov 05 06:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7046(8165493) done Nov 05 06:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2419(8165659) in scavenge on r911u05n02 Nov 05 06:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_481(8164034) WEXITSTATUS 0 Nov 05 06:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_481(8164034) done Nov 05 06:29:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=8130506 Nodelist=r808u14n01 Nov 05 06:29:30 r912u23n01 slurmctld[163760]: slurmctld: cleanup_completing: JobId=8130506 completion process took 108 seconds Nov 05 06:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6653(8164732) WEXITSTATUS 1 Nov 05 06:29:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6653(8164732) done Nov 05 06:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_473(8164026) WEXITSTATUS 0 Nov 05 06:29:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_473(8164026) done Nov 05 06:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1776(8157983) WEXITSTATUS 0 Nov 05 06:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1776(8157983) done Nov 05 06:29:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2420(8165660) NodeList=r909u29n01 #CPUs=1 Partition=day Nov 05 06:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1833(8158547) WEXITSTATUS 0 Nov 05 06:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1833(8158547) done Nov 05 06:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3819(8165211) WEXITSTATUS 0 Nov 05 06:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3819(8165211) done Nov 05 06:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_376(8163909) WEXITSTATUS 0 Nov 05 06:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_376(8163909) done Nov 05 06:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_495(8164060) WEXITSTATUS 0 Nov 05 06:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_495(8164060) done Nov 05 06:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_478(8164031) WEXITSTATUS 0 Nov 05 06:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_478(8164031) done Nov 05 06:29:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_314(8163815) WEXITSTATUS 0 Nov 05 06:29:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_314(8163815) done Nov 05 06:29:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_395(8163928) WEXITSTATUS 0 Nov 05 06:29:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_395(8163928) done Nov 05 06:29:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6784(8164938) WEXITSTATUS 1 Nov 05 06:29:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6784(8164938) done Nov 05 06:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_492(8164057) WEXITSTATUS 0 Nov 05 06:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_492(8164057) done Nov 05 06:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1818(8158430) WEXITSTATUS 0 Nov 05 06:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1818(8158430) done Nov 05 06:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_374(8163907) WEXITSTATUS 0 Nov 05 06:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_374(8163907) done Nov 05 06:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_364(8163885) WEXITSTATUS 0 Nov 05 06:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_364(8163885) done Nov 05 06:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_272(8163758) WEXITSTATUS 0 Nov 05 06:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_272(8163758) done Nov 05 06:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_358(8163879) WEXITSTATUS 0 Nov 05 06:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_358(8163879) done Nov 05 06:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_570(8164166) WEXITSTATUS 0 Nov 05 06:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_570(8164166) done Nov 05 06:30:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6933(8165216) WEXITSTATUS 1 Nov 05 06:30:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6933(8165216) done Nov 05 06:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1837(8158616) WEXITSTATUS 0 Nov 05 06:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1837(8158616) done Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_575(8164194) WEXITSTATUS 0 Nov 05 06:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_575(8164194) done Nov 05 06:30:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_686(7572855) Nov 05 06:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_275(8163761) WEXITSTATUS 0 Nov 05 06:30:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_275(8163761) done Nov 05 06:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_441(8163984) WEXITSTATUS 0 Nov 05 06:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_441(8163984) done Nov 05 06:30:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_211(8163663) WEXITSTATUS 0 Nov 05 06:30:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_211(8163663) done Nov 05 06:30:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 06:30:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 06:30:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2421(8165661) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 06:30:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2422(8165662) NodeList=r909u04n01 #CPUs=1 Partition=day Nov 05 06:30:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1866(8161213) WEXITSTATUS 0 Nov 05 06:30:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1866(8161213) done Nov 05 06:30:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1817(8158429) WEXITSTATUS 0 Nov 05 06:30:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1817(8158429) done Nov 05 06:30:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_566(8164162) WEXITSTATUS 0 Nov 05 06:30:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_566(8164162) done Nov 05 06:30:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_537(8164118) WEXITSTATUS 0 Nov 05 06:30:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_537(8164118) done Nov 05 06:30:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_310(8163811) WEXITSTATUS 0 Nov 05 06:30:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_310(8163811) done Nov 05 06:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 06:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 06:30:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 06:30:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_207(8163659) WEXITSTATUS 0 Nov 05 06:30:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_207(8163659) done Nov 05 06:30:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1752(8157781) WEXITSTATUS 0 Nov 05 06:30:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1752(8157781) done Nov 05 06:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_589(8164208) WEXITSTATUS 0 Nov 05 06:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_589(8164208) done Nov 05 06:31:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7050(8165497) WEXITSTATUS 1 Nov 05 06:31:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7050(8165497) done Nov 05 06:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_513(8164078) WEXITSTATUS 0 Nov 05 06:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_513(8164078) done Nov 05 06:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_438(8163981) WEXITSTATUS 0 Nov 05 06:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_438(8163981) done Nov 05 06:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6941(8165224) WEXITSTATUS 1 Nov 05 06:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6941(8165224) done Nov 05 06:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_2(8158748) WEXITSTATUS 0 Nov 05 06:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_2(8158748) done Nov 05 06:31:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_359(8163880) WEXITSTATUS 0 Nov 05 06:31:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_359(8163880) done Nov 05 06:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_476(8164029) WEXITSTATUS 0 Nov 05 06:31:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_476(8164029) done Nov 05 06:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3667(8162996) WEXITSTATUS 0 Nov 05 06:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3667(8162996) done Nov 05 06:31:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1687(8156581) WEXITSTATUS 0 Nov 05 06:31:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1687(8156581) done Nov 05 06:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2641(8153028) WEXITSTATUS 0 Nov 05 06:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2641(8153028) done Nov 05 06:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6465(8164370) WEXITSTATUS 1 Nov 05 06:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6465(8164370) done Nov 05 06:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_3(8158163) WEXITSTATUS 1 Nov 05 06:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_3(8158163) done Nov 05 06:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_435(8163978) WEXITSTATUS 0 Nov 05 06:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_435(8163978) done Nov 05 06:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2423(8165663) NodeList=r909u01n03 #CPUs=1 Partition=day Nov 05 06:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2424(8165664) NodeList=r909u04n04 #CPUs=1 Partition=day Nov 05 06:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2425(8165665) NodeList=r909u07n02 #CPUs=1 Partition=day Nov 05 06:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2426(8165666) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 06:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2427(8165667) NodeList=r815u29n01 #CPUs=1 Partition=day Nov 05 06:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3839(8165668) in gpu on r808u08n01 Nov 05 06:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3840(8165669) in gpu on r808u14n01 Nov 05 06:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_454(8164007) WEXITSTATUS 0 Nov 05 06:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_454(8164007) done Nov 05 06:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7103(8165670) in scavenge on r907u05n03 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7104(8165671) in scavenge on r907u09n03 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7105(8165672) in scavenge on r815u03n12 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7106(8165673) in scavenge on r816u23n12 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7107(8165674) in scavenge on r816u35n10 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7108(8165675) in scavenge on r808u17n01 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7109(8165676) in scavenge on r805u25n03 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_737(7572906) in scavenge on r917u13n03 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_805(7572974) in scavenge on r917u29n04 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_827(7572996) in scavenge on r815u23n12 Nov 05 06:31:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_835(7573004) in scavenge on r904u04n04 Nov 05 06:31:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_350(8163871) WEXITSTATUS 0 Nov 05 06:31:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_350(8163871) done Nov 05 06:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3840(8165669) WEXITSTATUS 0 Nov 05 06:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3840(8165669) done Nov 05 06:31:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2428(8165677) in scavenge on r911u05n01 Nov 05 06:31:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2429(8165678) in scavenge on r917u13n02 Nov 05 06:32:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_439(8163982) WEXITSTATUS 0 Nov 05 06:32:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_439(8163982) done Nov 05 06:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6877(8165107) WEXITSTATUS 1 Nov 05 06:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6877(8165107) done Nov 05 06:32:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_345(8163866) WEXITSTATUS 0 Nov 05 06:32:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_345(8163866) done Nov 05 06:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_322(8163823) WEXITSTATUS 0 Nov 05 06:32:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_322(8163823) done Nov 05 06:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1734(8157684) WEXITSTATUS 0 Nov 05 06:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1734(8157684) done Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_531(8164112) WEXITSTATUS 0 Nov 05 06:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_531(8164112) done Nov 05 06:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_530(8164111) WEXITSTATUS 0 Nov 05 06:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_530(8164111) done Nov 05 06:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1700(8156845) WEXITSTATUS 0 Nov 05 06:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1700(8156845) done Nov 05 06:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7104(8165671) WEXITSTATUS 1 Nov 05 06:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7104(8165671) done Nov 05 06:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_418(8163961) WEXITSTATUS 0 Nov 05 06:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_418(8163961) done Nov 05 06:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2430(8165679) NodeList=r918u05n01 #CPUs=1 Partition=day Nov 05 06:32:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2431(8165680) NodeList=r919u05n02 #CPUs=1 Partition=day Nov 05 06:32:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_426(8163969) WEXITSTATUS 0 Nov 05 06:32:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_426(8163969) done Nov 05 06:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_335(8163856) WEXITSTATUS 0 Nov 05 06:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_335(8163856) done Nov 05 06:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_387(8163920) WEXITSTATUS 0 Nov 05 06:32:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_387(8163920) done Nov 05 06:32:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1722(8157328) WEXITSTATUS 0 Nov 05 06:32:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1722(8157328) done Nov 05 06:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_489(8164054) WEXITSTATUS 0 Nov 05 06:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_489(8164054) done Nov 05 06:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_427(8163970) WEXITSTATUS 0 Nov 05 06:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_427(8163970) done Nov 05 06:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6937(8165220) WEXITSTATUS 1 Nov 05 06:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6937(8165220) done Nov 05 06:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6794(8164950) WEXITSTATUS 1 Nov 05 06:32:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6794(8164950) done Nov 05 06:32:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2597(8152864) WEXITSTATUS 0 Nov 05 06:32:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2597(8152864) done Nov 05 06:32:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_372(8114639) Nov 05 06:32:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:32:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 06:32:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 06:32:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 06:32:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6697(8164802) WEXITSTATUS 1 Nov 05 06:32:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6697(8164802) done Nov 05 06:33:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1864(8160862) WEXITSTATUS 0 Nov 05 06:33:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1864(8160862) done Nov 05 06:33:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_563(8164159) WEXITSTATUS 0 Nov 05 06:33:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_563(8164159) done Nov 05 06:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_14(8158175) WEXITSTATUS 1 Nov 05 06:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_14(8158175) done Nov 05 06:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_493(8164058) WEXITSTATUS 0 Nov 05 06:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_493(8164058) done Nov 05 06:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4482(8155855) WEXITSTATUS 1 Nov 05 06:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4482(8155855) done Nov 05 06:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6120(8163161) WEXITSTATUS 1 Nov 05 06:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6120(8163161) done Nov 05 06:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_378(8163911) WEXITSTATUS 0 Nov 05 06:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_378(8163911) done Nov 05 06:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6965(8165285) WEXITSTATUS 1 Nov 05 06:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6965(8165285) done Nov 05 06:33:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7058(8165521) WEXITSTATUS 1 Nov 05 06:33:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7058(8165521) done Nov 05 06:33:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_488(8164053) WEXITSTATUS 0 Nov 05 06:33:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_488(8164053) done Nov 05 06:33:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2432(8165681) NodeList=r918u09n04 #CPUs=1 Partition=day Nov 05 06:33:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2433(8165682) NodeList=r815u09n02 #CPUs=1 Partition=day Nov 05 06:33:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2434(8165683) NodeList=r816u03n07 #CPUs=1 Partition=day Nov 05 06:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1692(8156714) WEXITSTATUS 0 Nov 05 06:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1692(8156714) done Nov 05 06:33:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_605(8164249) WEXITSTATUS 0 Nov 05 06:33:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_605(8164249) done Nov 05 06:33:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_380(8163913) WEXITSTATUS 0 Nov 05 06:33:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_380(8163913) done Nov 05 06:33:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6807(8164977) WEXITSTATUS 1 Nov 05 06:33:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6807(8164977) done Nov 05 06:33:46 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:33:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:33:48 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 06:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1666(8156299) WEXITSTATUS 0 Nov 05 06:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1666(8156299) done Nov 05 06:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6773(8164925) WEXITSTATUS 1 Nov 05 06:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6773(8164925) done Nov 05 06:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_586(8164205) WEXITSTATUS 0 Nov 05 06:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_586(8164205) done Nov 05 06:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_334(8163855) WEXITSTATUS 0 Nov 05 06:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_334(8163855) done Nov 05 06:34:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_610(8164254) WEXITSTATUS 0 Nov 05 06:34:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_610(8164254) done Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820341 WEXITSTATUS 0 Nov 05 06:34:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820341 done Nov 05 06:34:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820370 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 06:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_519(8164084) WEXITSTATUS 0 Nov 05 06:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_519(8164084) done Nov 05 06:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2030(8151590) WEXITSTATUS 0 Nov 05 06:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2030(8151590) done Nov 05 06:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_286(8163772) WEXITSTATUS 0 Nov 05 06:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_286(8163772) done Nov 05 06:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_434(8163977) WEXITSTATUS 0 Nov 05 06:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_434(8163977) done Nov 05 06:34:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_609(8164253) WEXITSTATUS 0 Nov 05 06:34:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_609(8164253) done Nov 05 06:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_558(8164154) WEXITSTATUS 0 Nov 05 06:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_558(8164154) done Nov 05 06:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1829(8158508) WEXITSTATUS 0 Nov 05 06:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1829(8158508) done Nov 05 06:34:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3841(8165684) in gpu on r808u14n01 Nov 05 06:34:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2435(8165685) in day on r806u02n03 Nov 05 06:34:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2436(8165686) in day on r918u09n01 Nov 05 06:34:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2437(8165687) in day on r919u09n03 Nov 05 06:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7097(8165653) WEXITSTATUS 1 Nov 05 06:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7097(8165653) done Nov 05 06:34:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165688 InitPrio=20041 usec=4063 Nov 05 06:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165440 WEXITSTATUS 0 Nov 05 06:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165440 done Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7110(8165689) in scavenge on r801u30n02 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7111(8165690) in scavenge on r907u09n03 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7112(8165691) in scavenge on r903u07n01 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7113(8165692) in scavenge on r911u01n01 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7114(8165693) in scavenge on r916u04n02 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7115(8165694) in scavenge on r917u13n02 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7116(8165695) in scavenge on r917u29n03 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7117(8165696) in scavenge on r815u29n08 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7118(8165697) in scavenge on r816u15n10 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7119(8165698) in scavenge on r906u28n01 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7120(8165699) in scavenge on r806u02n02 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7121(8165700) in scavenge on r806u02n02 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_752(7572921) in scavenge on r917u13n01 Nov 05 06:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_770(7572939) in scavenge on r815u23n09 Nov 05 06:34:37 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 06:34:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165688 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 #CPUs=288 Partition=day Nov 05 06:34:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6473(8164380) WEXITSTATUS 1 Nov 05 06:34:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6473(8164380) done Nov 05 06:34:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_301(8163802) WEXITSTATUS 0 Nov 05 06:34:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_301(8163802) done Nov 05 06:34:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_477(8164030) WEXITSTATUS 0 Nov 05 06:34:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_477(8164030) done Nov 05 06:34:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_475(8164028) WEXITSTATUS 0 Nov 05 06:34:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_475(8164028) done Nov 05 06:34:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_542(8164123) WEXITSTATUS 0 Nov 05 06:34:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_542(8164123) done Nov 05 06:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6608(8164663) WEXITSTATUS 1 Nov 05 06:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6608(8164663) done Nov 05 06:35:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1862(8160523) WEXITSTATUS 0 Nov 05 06:35:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1862(8160523) done Nov 05 06:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2554(8152821) WEXITSTATUS 0 Nov 05 06:35:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2554(8152821) done Nov 05 06:35:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_450(8164003) WEXITSTATUS 0 Nov 05 06:35:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_450(8164003) done Nov 05 06:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_527(8164108) WEXITSTATUS 0 Nov 05 06:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_527(8164108) done Nov 05 06:35:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_204(8163656) WEXITSTATUS 0 Nov 05 06:35:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_204(8163656) done Nov 05 06:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7030(8165455) WEXITSTATUS 1 Nov 05 06:35:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7030(8165455) done Nov 05 06:35:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6894(8165141) WEXITSTATUS 1 Nov 05 06:35:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6894(8165141) done Nov 05 06:35:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_743(7572912) Nov 05 06:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1683(8156457) WEXITSTATUS 0 Nov 05 06:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1683(8156457) done Nov 05 06:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6891(8165130) WEXITSTATUS 1 Nov 05 06:35:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6891(8165130) done Nov 05 06:35:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2438(8165701) NodeList=r805u02n03 #CPUs=1 Partition=day Nov 05 06:35:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6310(8163706) WEXITSTATUS 1 Nov 05 06:35:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6310(8163706) done Nov 05 06:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6839(8165034) WEXITSTATUS 1 Nov 05 06:35:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6839(8165034) done Nov 05 06:35:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_612(8164256) WEXITSTATUS 0 Nov 05 06:35:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_612(8164256) done Nov 05 06:35:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_270(8163756) WEXITSTATUS 0 Nov 05 06:35:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_270(8163756) done Nov 05 06:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_511(8164076) WEXITSTATUS 0 Nov 05 06:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_511(8164076) done Nov 05 06:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_459(8164012) WEXITSTATUS 0 Nov 05 06:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_459(8164012) done Nov 05 06:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_510(8164075) WEXITSTATUS 0 Nov 05 06:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_510(8164075) done Nov 05 06:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6312(8163708) WEXITSTATUS 1 Nov 05 06:35:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6312(8163708) done Nov 05 06:36:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_502(8164067) WEXITSTATUS 0 Nov 05 06:36:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_502(8164067) done Nov 05 06:36:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_535(8164116) WEXITSTATUS 0 Nov 05 06:36:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_535(8164116) done Nov 05 06:36:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_437(8163980) WEXITSTATUS 0 Nov 05 06:36:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_437(8163980) done Nov 05 06:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_415(8163958) WEXITSTATUS 0 Nov 05 06:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_415(8163958) done Nov 05 06:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6745(8164870) WEXITSTATUS 1 Nov 05 06:36:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6745(8164870) done Nov 05 06:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1819(8158431) WEXITSTATUS 0 Nov 05 06:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1819(8158431) done Nov 05 06:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1716(8157210) WEXITSTATUS 0 Nov 05 06:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1716(8157210) done Nov 05 06:36:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_532(8164113) WEXITSTATUS 0 Nov 05 06:36:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_532(8164113) done Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_449(8164002) WEXITSTATUS 0 Nov 05 06:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_449(8164002) done Nov 05 06:36:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2439(8165702) NodeList=r909u04n02 #CPUs=1 Partition=day Nov 05 06:36:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2440(8165703) NodeList=r909u29n04 #CPUs=1 Partition=day Nov 05 06:36:41 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 06:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_379(8163912) WEXITSTATUS 0 Nov 05 06:36:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_379(8163912) done Nov 05 06:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_397(8163930) WEXITSTATUS 0 Nov 05 06:36:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_397(8163930) done Nov 05 06:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_564(8164160) WEXITSTATUS 0 Nov 05 06:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_564(8164160) done Nov 05 06:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6857(8165066) WEXITSTATUS 1 Nov 05 06:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6857(8165066) done Nov 05 06:36:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_240(8163710) WEXITSTATUS 0 Nov 05 06:36:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_240(8163710) done Nov 05 06:36:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3841(8165684) WEXITSTATUS 0 Nov 05 06:36:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3841(8165684) done Nov 05 06:37:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_401(8163934) WEXITSTATUS 0 Nov 05 06:37:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_401(8163934) done Nov 05 06:37:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3842(8165704) in gpu on r808u14n01 Nov 05 06:37:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_594(8164238) WEXITSTATUS 0 Nov 05 06:37:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_594(8164238) done Nov 05 06:37:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_507(8164072) WEXITSTATUS 0 Nov 05 06:37:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_507(8164072) done Nov 05 06:37:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7122(8165705) in scavenge on r802u02n03 Nov 05 06:37:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_516(8164081) WEXITSTATUS 0 Nov 05 06:37:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_516(8164081) done Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7123(8165706) in scavenge on r915u35n02 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7124(8165707) in scavenge on r815u03n12 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7125(8165708) in scavenge on r815u23n08 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7126(8165709) in scavenge on r808u14n01 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7127(8165710) in scavenge on r806u04n04 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7128(8165711) in scavenge on r807u11n01 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7129(8165712) in scavenge on r904u10n01 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7130(8165713) in scavenge on r914u01n01 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7131(8165714) in scavenge on r805u04n02 Nov 05 06:37:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_99(8114365) in scavenge on r805u25n01 Nov 05 06:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6766(8164915) WEXITSTATUS 1 Nov 05 06:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6766(8164915) done Nov 05 06:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_425(8163968) WEXITSTATUS 0 Nov 05 06:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_425(8163968) done Nov 05 06:37:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2441(8165715) in scavenge on r917u29n03 Nov 05 06:37:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2442(8165716) in scavenge on r816u23n08 Nov 05 06:37:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2443(8165717) in scavenge on r808u14n01 Nov 05 06:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_524(8164105) WEXITSTATUS 0 Nov 05 06:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_524(8164105) done Nov 05 06:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_593(8164212) WEXITSTATUS 0 Nov 05 06:37:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_593(8164212) done Nov 05 06:37:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_614(8164278) WEXITSTATUS 0 Nov 05 06:37:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_614(8164278) done Nov 05 06:37:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_515(8164080) WEXITSTATUS 0 Nov 05 06:37:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_515(8164080) done Nov 05 06:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7130(8165713) WEXITSTATUS 1 Nov 05 06:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7130(8165713) done Nov 05 06:37:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_587(8164206) WEXITSTATUS 0 Nov 05 06:37:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_587(8164206) done Nov 05 06:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6597(8164639) WEXITSTATUS 1 Nov 05 06:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6597(8164639) done Nov 05 06:37:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_598(8164242) WEXITSTATUS 0 Nov 05 06:37:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_598(8164242) done Nov 05 06:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_549(8164145) WEXITSTATUS 0 Nov 05 06:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_549(8164145) done Nov 05 06:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1779(8157986) WEXITSTATUS 0 Nov 05 06:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1779(8157986) done Nov 05 06:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_382(8163915) WEXITSTATUS 0 Nov 05 06:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_382(8163915) done Nov 05 06:37:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_545(8164126) WEXITSTATUS 0 Nov 05 06:37:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_545(8164126) done Nov 05 06:38:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_592(8164211) WEXITSTATUS 0 Nov 05 06:38:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_592(8164211) done Nov 05 06:38:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_618(8164282) WEXITSTATUS 0 Nov 05 06:38:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_618(8164282) done Nov 05 06:38:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_480(8164033) WEXITSTATUS 0 Nov 05 06:38:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_480(8164033) done Nov 05 06:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_336(8163857) WEXITSTATUS 0 Nov 05 06:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_336(8163857) done Nov 05 06:38:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1743(8157746) WEXITSTATUS 0 Nov 05 06:38:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1743(8157746) done Nov 05 06:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_517(8164082) WEXITSTATUS 0 Nov 05 06:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_517(8164082) done Nov 05 06:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6181(8163267) WEXITSTATUS 1 Nov 05 06:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6181(8163267) done Nov 05 06:38:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_467(8164020) WEXITSTATUS 0 Nov 05 06:38:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_467(8164020) done Nov 05 06:38:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6814(8164986) WEXITSTATUS 1 Nov 05 06:38:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6814(8164986) done Nov 05 06:38:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_398(8163931) WEXITSTATUS 0 Nov 05 06:38:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_398(8163931) done Nov 05 06:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820338 WEXITSTATUS 0 Nov 05 06:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820338 done Nov 05 06:38:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820371 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:38:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:38:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1826(8158482) WEXITSTATUS 0 Nov 05 06:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1826(8158482) done Nov 05 06:38:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_472(8164025) WEXITSTATUS 0 Nov 05 06:38:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_472(8164025) done Nov 05 06:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_440(8163983) WEXITSTATUS 0 Nov 05 06:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_440(8163983) done Nov 05 06:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_607(8164251) WEXITSTATUS 0 Nov 05 06:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_607(8164251) done Nov 05 06:38:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_497(8164062) WEXITSTATUS 0 Nov 05 06:38:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_497(8164062) done Nov 05 06:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_557(8164153) WEXITSTATUS 0 Nov 05 06:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_557(8164153) done Nov 05 06:38:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2444(8165718) NodeList=r806u06n03 #CPUs=1 Partition=day Nov 05 06:38:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2445(8165719) NodeList=r909u07n04 #CPUs=1 Partition=day Nov 05 06:38:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2446(8165720) NodeList=r919u01n02 #CPUs=1 Partition=day Nov 05 06:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_567(8164163) WEXITSTATUS 0 Nov 05 06:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_567(8164163) done Nov 05 06:38:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_384(8163917) WEXITSTATUS 0 Nov 05 06:38:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_384(8163917) done Nov 05 06:38:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_505(8164070) WEXITSTATUS 0 Nov 05 06:38:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_505(8164070) done Nov 05 06:38:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_601(8164245) WEXITSTATUS 0 Nov 05 06:38:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_601(8164245) done Nov 05 06:38:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_67(8135766) WEXITSTATUS 0 Nov 05 06:38:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_67(8135766) done Nov 05 06:38:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_559(8164155) WEXITSTATUS 0 Nov 05 06:38:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_559(8164155) done Nov 05 06:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7114(8165693) WEXITSTATUS 1 Nov 05 06:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7114(8165693) done Nov 05 06:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_508(8164073) WEXITSTATUS 0 Nov 05 06:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_508(8164073) done Nov 05 06:39:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_469(8164022) WEXITSTATUS 0 Nov 05 06:39:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_469(8164022) done Nov 05 06:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_451(8164004) WEXITSTATUS 0 Nov 05 06:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_451(8164004) done Nov 05 06:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159929 WEXITSTATUS 0 Nov 05 06:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159929 done Nov 05 06:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6955(8165275) WEXITSTATUS 1 Nov 05 06:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6955(8165275) done Nov 05 06:39:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6725(8164840) WEXITSTATUS 1 Nov 05 06:39:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6725(8164840) done Nov 05 06:39:28 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r4513u26n01 reason set to: NHC: check_cmd_status: 142 (expected 0) returned by "test -e /vast/palmer/apps/.yalehpc". Nov 05 06:39:28 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r4513u26n01 state set to DRAINING Nov 05 06:39:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_300(8163801) WEXITSTATUS 0 Nov 05 06:39:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_300(8163801) done Nov 05 06:39:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 06:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_12(8158173) WEXITSTATUS 1 Nov 05 06:39:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_12(8158173) done Nov 05 06:39:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_490(8164055) WEXITSTATUS 0 Nov 05 06:39:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_490(8164055) done Nov 05 06:39:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_525(8164106) WEXITSTATUS 0 Nov 05 06:39:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_525(8164106) done Nov 05 06:39:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_11(8158172) WEXITSTATUS 1 Nov 05 06:39:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_11(8158172) done Nov 05 06:39:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2447(8165721) NodeList=r914u01n03 #CPUs=1 Partition=day Nov 05 06:39:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2448(8165722) NodeList=r815u03n03 #CPUs=1 Partition=day Nov 05 06:39:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2449(8165723) NodeList=r815u03n05 #CPUs=1 Partition=day Nov 05 06:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_573(8164192) WEXITSTATUS 0 Nov 05 06:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_573(8164192) done Nov 05 06:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6896(8165143) WEXITSTATUS 1 Nov 05 06:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6896(8165143) done Nov 05 06:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1121(8131613) WEXITSTATUS 0 Nov 05 06:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1121(8131613) done Nov 05 06:39:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_561(8164157) WEXITSTATUS 0 Nov 05 06:39:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_561(8164157) done Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7132(8165724) in scavenge on r801u06n01 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7133(8165725) in scavenge on r802u04n03 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7134(8165726) in scavenge on r916u04n02 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7135(8165727) in scavenge on r917u13n01 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7136(8165728) in scavenge on r917u13n03 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7137(8165729) in scavenge on r917u29n01 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7138(8165730) in scavenge on r917u29n03 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7139(8165731) in scavenge on r917u29n04 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7140(8165732) in scavenge on r815u23n08 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_56(7572223) in scavenge on r816u29n07 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_618(7572787) in scavenge on r807u05n01 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_725(7572894) in scavenge on r815u03n05 Nov 05 06:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_215(8114481) in scavenge on r815u03n05 Nov 05 06:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_396(8163929) WEXITSTATUS 0 Nov 05 06:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_396(8163929) done Nov 05 06:40:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2450(8165733) in scavenge on r917u13n03 Nov 05 06:40:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2451(8165734) in scavenge on r816u29n07 Nov 05 06:40:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2452(8165735) in scavenge on r907u32n01 Nov 05 06:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_443(8163986) WEXITSTATUS 0 Nov 05 06:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_443(8163986) done Nov 05 06:40:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_457(8164010) WEXITSTATUS 0 Nov 05 06:40:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_457(8164010) done Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:40:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_595(8164239) WEXITSTATUS 0 Nov 05 06:40:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_595(8164239) done Nov 05 06:40:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_423(8163966) WEXITSTATUS 0 Nov 05 06:40:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_423(8163966) done Nov 05 06:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_333(8163854) WEXITSTATUS 0 Nov 05 06:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_333(8163854) done Nov 05 06:40:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_498(8164063) WEXITSTATUS 0 Nov 05 06:40:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_498(8164063) done Nov 05 06:40:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6693(8164795) WEXITSTATUS 1 Nov 05 06:40:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6693(8164795) done Nov 05 06:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_538(8164119) WEXITSTATUS 0 Nov 05 06:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_538(8164119) done Nov 05 06:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_615(8164279) WEXITSTATUS 0 Nov 05 06:40:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_615(8164279) done Nov 05 06:40:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8157220 Nov 05 06:40:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_453(8164006) WEXITSTATUS 0 Nov 05 06:40:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_453(8164006) done Nov 05 06:40:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5241(8158648) WEXITSTATUS 1 Nov 05 06:40:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5241(8158648) done Nov 05 06:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7086(8165627) WEXITSTATUS 1 Nov 05 06:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7086(8165627) done Nov 05 06:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1803(8158199) WEXITSTATUS 0 Nov 05 06:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1803(8158199) done Nov 05 06:41:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_381(8163914) WEXITSTATUS 0 Nov 05 06:41:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_381(8163914) done Nov 05 06:41:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_596(8164240) WEXITSTATUS 0 Nov 05 06:41:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_596(8164240) done Nov 05 06:41:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6792(8164948) WEXITSTATUS 1 Nov 05 06:41:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6792(8164948) done Nov 05 06:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_529(8164110) WEXITSTATUS 0 Nov 05 06:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_529(8164110) done Nov 05 06:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_528(8164109) WEXITSTATUS 0 Nov 05 06:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_528(8164109) done Nov 05 06:41:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_534(8164115) WEXITSTATUS 0 Nov 05 06:41:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_534(8164115) done Nov 05 06:41:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820339 WEXITSTATUS 0 Nov 05 06:41:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820339 done Nov 05 06:41:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820372 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 06:41:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7103(8165670) WEXITSTATUS 1 Nov 05 06:41:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7103(8165670) done Nov 05 06:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_552(8164148) WEXITSTATUS 0 Nov 05 06:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_552(8164148) done Nov 05 06:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_536(8164117) WEXITSTATUS 0 Nov 05 06:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_536(8164117) done Nov 05 06:41:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7136(8165728) WEXITSTATUS 1 Nov 05 06:41:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7136(8165728) done Nov 05 06:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_571(8164190) WEXITSTATUS 0 Nov 05 06:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_571(8164190) done Nov 05 06:41:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_348(8163869) WEXITSTATUS 0 Nov 05 06:41:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_348(8163869) done Nov 05 06:41:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_578(8164197) WEXITSTATUS 0 Nov 05 06:41:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_578(8164197) done Nov 05 06:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_583(8164202) WEXITSTATUS 0 Nov 05 06:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_583(8164202) done Nov 05 06:42:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144136 WEXITSTATUS 0 Nov 05 06:42:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144136 done Nov 05 06:42:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_506(8164071) WEXITSTATUS 0 Nov 05 06:42:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_506(8164071) done Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:42:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:42:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:42:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144157 in scavenge on r915u38n02,r917u04n01 Nov 05 06:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_603(8164247) WEXITSTATUS 0 Nov 05 06:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_603(8164247) done Nov 05 06:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6286(8163605) WEXITSTATUS 1 Nov 05 06:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6286(8163605) done Nov 05 06:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_617(8164281) WEXITSTATUS 0 Nov 05 06:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_617(8164281) done Nov 05 06:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_422(8163965) WEXITSTATUS 0 Nov 05 06:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_422(8163965) done Nov 05 06:42:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7141(8165736) in scavenge on r907u01n04 Nov 05 06:42:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7142(8165737) in scavenge on r907u05n03 Nov 05 06:42:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7143(8165738) in scavenge on r903u01n01 Nov 05 06:42:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7144(8165739) in scavenge on r917u13n03 Nov 05 06:42:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7145(8165740) in scavenge on r805u25n04 Nov 05 06:42:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7146(8165741) in scavenge on r807u11n01 Nov 05 06:42:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7147(8165742) in scavenge on r914u01n01 Nov 05 06:42:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_63(7572230) in scavenge on r801u32n02 Nov 05 06:42:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_574(8164193) WEXITSTATUS 0 Nov 05 06:42:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_574(8164193) done Nov 05 06:42:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_554(8164150) WEXITSTATUS 0 Nov 05 06:42:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_554(8164150) done Nov 05 06:42:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2453(8165743) in scavenge on r917u29n04 Nov 05 06:42:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2454(8165744) in scavenge on r906u13n01 Nov 05 06:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6878(8165108) WEXITSTATUS 1 Nov 05 06:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6878(8165108) done Nov 05 06:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6816(8164988) WEXITSTATUS 1 Nov 05 06:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6816(8164988) done Nov 05 06:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_553(8164149) WEXITSTATUS 0 Nov 05 06:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_553(8164149) done Nov 05 06:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_576(8164195) WEXITSTATUS 0 Nov 05 06:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_576(8164195) done Nov 05 06:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3842(8165704) WEXITSTATUS 0 Nov 05 06:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3842(8165704) done Nov 05 06:42:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_533(8164114) WEXITSTATUS 0 Nov 05 06:42:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_533(8164114) done Nov 05 06:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3805(8165009) WEXITSTATUS 0 Nov 05 06:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3805(8165009) done Nov 05 06:43:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1718(8157212) WEXITSTATUS 0 Nov 05 06:43:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1718(8157212) done Nov 05 06:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1793(8158104) WEXITSTATUS 0 Nov 05 06:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1793(8158104) done Nov 05 06:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_522(8164103) WEXITSTATUS 0 Nov 05 06:43:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_522(8164103) done Nov 05 06:43:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6922(8165182) WEXITSTATUS 1 Nov 05 06:43:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6922(8165182) done Nov 05 06:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_579(8164198) WEXITSTATUS 0 Nov 05 06:43:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_579(8164198) done Nov 05 06:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_509(8164074) WEXITSTATUS 0 Nov 05 06:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_509(8164074) done Nov 05 06:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2455(8165745) NodeList=r909u10n03 #CPUs=1 Partition=day Nov 05 06:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2456(8165746) NodeList=r919u36n01 #CPUs=1 Partition=day Nov 05 06:43:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_6(8158166) WEXITSTATUS 1 Nov 05 06:43:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_6(8158166) done Nov 05 06:43:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_460(8164013) WEXITSTATUS 0 Nov 05 06:43:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_460(8164013) done Nov 05 06:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_543(8164124) WEXITSTATUS 0 Nov 05 06:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_543(8164124) done Nov 05 06:43:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:43:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_347(8163868) WEXITSTATUS 0 Nov 05 06:43:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_347(8163868) done Nov 05 06:43:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_8(8158266) WEXITSTATUS 1 Nov 05 06:43:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_8(8158266) done Nov 05 06:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_551(8164147) WEXITSTATUS 0 Nov 05 06:44:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_551(8164147) done Nov 05 06:44:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6932(8165215) WEXITSTATUS 1 Nov 05 06:44:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6932(8165215) done Nov 05 06:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1691(8156713) WEXITSTATUS 0 Nov 05 06:44:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1691(8156713) done Nov 05 06:44:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1853(8159760) WEXITSTATUS 0 Nov 05 06:44:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1853(8159760) done Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_621(8164285) WEXITSTATUS 0 Nov 05 06:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_621(8164285) done Nov 05 06:44:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6594(8164636) WEXITSTATUS 1 Nov 05 06:44:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6594(8164636) done Nov 05 06:44:27 r912u23n01 slurmctld[163760]: slurmctld: update_node: node r4513u26n01 state set to ALLOCATED Nov 05 06:44:28 r912u23n01 slurmctld[163760]: slurmctld: Node r4513u26n01 now responding Nov 05 06:44:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 06:44:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_471(8164024) WEXITSTATUS 0 Nov 05 06:44:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_471(8164024) done Nov 05 06:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1875(8161846) WEXITSTATUS 0 Nov 05 06:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1875(8161846) done Nov 05 06:44:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_591(8164210) WEXITSTATUS 0 Nov 05 06:44:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_591(8164210) done Nov 05 06:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7023(8165448) WEXITSTATUS 1 Nov 05 06:44:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7023(8165448) done Nov 05 06:44:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2457(8165747) NodeList=r909u07n01 #CPUs=1 Partition=day Nov 05 06:44:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2458(8165748) NodeList=r816u09n07 #CPUs=1 Partition=day Nov 05 06:44:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2459(8165749) NodeList=r815u03n02 #CPUs=1 Partition=day Nov 05 06:44:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2460(8165750) NodeList=r815u03n08 #CPUs=1 Partition=day Nov 05 06:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_499(8164064) WEXITSTATUS 0 Nov 05 06:44:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_499(8164064) done Nov 05 06:44:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_577(8164196) WEXITSTATUS 0 Nov 05 06:44:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_577(8164196) done Nov 05 06:44:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_416(8163959) WEXITSTATUS 0 Nov 05 06:44:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_416(8163959) done Nov 05 06:45:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_572(8164191) WEXITSTATUS 0 Nov 05 06:45:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_572(8164191) done Nov 05 06:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6962(8165282) WEXITSTATUS 1 Nov 05 06:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6962(8165282) done Nov 05 06:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_547(8164143) WEXITSTATUS 0 Nov 05 06:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_547(8164143) done Nov 05 06:45:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3843(8165751) in gpu on r808u14n01 Nov 05 06:45:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3844(8165752) in gpu on r4513u29n01 Nov 05 06:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7109(8165676) WEXITSTATUS 1 Nov 05 06:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7109(8165676) done Nov 05 06:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6762(8164898) WEXITSTATUS 1 Nov 05 06:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6762(8164898) done Nov 05 06:45:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_608(8164252) WEXITSTATUS 0 Nov 05 06:45:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_608(8164252) done Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_744(7572913) Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7148(8165753) in scavenge on r907u09n03 Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7149(8165754) in scavenge on r916u01n01 Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7150(8165755) in scavenge on r917u13n03 Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7151(8165756) in scavenge on r917u29n01 Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7152(8165757) in scavenge on r815u09n11 Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7153(8165758) in scavenge on r816u23n11 Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7154(8165759) in scavenge on r805u14n03 Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7155(8165760) in scavenge on r904u04n02 Nov 05 06:45:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7156(8165761) in scavenge on r914u01n03 Nov 05 06:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_523(8164104) WEXITSTATUS 0 Nov 05 06:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_523(8164104) done Nov 05 06:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2419(8152463) WEXITSTATUS 0 Nov 05 06:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2419(8152463) done Nov 05 06:45:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2461(8165762) in scavenge on r917u13n01 Nov 05 06:45:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_546(8164127) WEXITSTATUS 0 Nov 05 06:45:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_546(8164127) done Nov 05 06:45:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162667 WEXITSTATUS 0 Nov 05 06:45:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162667 done Nov 05 06:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3843(8165751) WEXITSTATUS 0 Nov 05 06:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3843(8165751) done Nov 05 06:45:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2462(8165763) NodeList=r805u11n01 #CPUs=1 Partition=day Nov 05 06:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3844(8165752) WEXITSTATUS 0 Nov 05 06:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3844(8165752) done Nov 05 06:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_613(8164257) WEXITSTATUS 0 Nov 05 06:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_613(8164257) done Nov 05 06:45:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6883(8165113) WEXITSTATUS 1 Nov 05 06:45:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6883(8165113) done Nov 05 06:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6953(8165245) WEXITSTATUS 1 Nov 05 06:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6953(8165245) done Nov 05 06:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1778(8157985) WEXITSTATUS 0 Nov 05 06:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1778(8157985) done Nov 05 06:46:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_372(8163905) WEXITSTATUS 0 Nov 05 06:46:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_372(8163905) done Nov 05 06:46:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_479(8164032) WEXITSTATUS 0 Nov 05 06:46:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_479(8164032) done Nov 05 06:46:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6942(8165225) WEXITSTATUS 1 Nov 05 06:46:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6942(8165225) done Nov 05 06:46:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_584(8164203) WEXITSTATUS 0 Nov 05 06:46:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_584(8164203) done Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:46:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:46:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_600(8164244) WEXITSTATUS 0 Nov 05 06:46:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_600(8164244) done Nov 05 06:46:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1741(8157744) WEXITSTATUS 0 Nov 05 06:46:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1741(8157744) done Nov 05 06:46:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7060(8165523) WEXITSTATUS 1 Nov 05 06:46:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7060(8165523) done Nov 05 06:46:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2463(8165764) NodeList=r914u05n02 #CPUs=1 Partition=day Nov 05 06:46:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2464(8165765) NodeList=r918u32n01 #CPUs=1 Partition=day Nov 05 06:46:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6853(8165062) WEXITSTATUS 1 Nov 05 06:46:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6853(8165062) done Nov 05 06:46:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_373(8163906) WEXITSTATUS 0 Nov 05 06:46:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_373(8163906) done Nov 05 06:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_560(8164156) WEXITSTATUS 0 Nov 05 06:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_560(8164156) done Nov 05 06:46:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6967(8165287) WEXITSTATUS 1 Nov 05 06:46:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6967(8165287) done Nov 05 06:46:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_585(8164204) WEXITSTATUS 0 Nov 05 06:46:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_585(8164204) done Nov 05 06:47:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_599(8164243) WEXITSTATUS 0 Nov 05 06:47:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_599(8164243) done Nov 05 06:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_521(8164102) WEXITSTATUS 0 Nov 05 06:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_521(8164102) done Nov 05 06:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_466(8164019) WEXITSTATUS 0 Nov 05 06:47:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_466(8164019) done Nov 05 06:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_622(8164286) WEXITSTATUS 0 Nov 05 06:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_622(8164286) done Nov 05 06:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_597(8164241) WEXITSTATUS 0 Nov 05 06:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_597(8164241) done Nov 05 06:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_555(8164151) WEXITSTATUS 0 Nov 05 06:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_555(8164151) done Nov 05 06:47:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6900(8165147) WEXITSTATUS 1 Nov 05 06:47:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6900(8165147) done Nov 05 06:47:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6914(8165169) WEXITSTATUS 1 Nov 05 06:47:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6914(8165169) done Nov 05 06:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3845(8165766) in gpu on r808u14n01 Nov 05 06:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3846(8165767) in gpu on r4513u29n01 Nov 05 06:48:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:48:08 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7157(8165768) in scavenge on r917u29n02 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7158(8165769) in scavenge on r917u29n02 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7159(8165770) in scavenge on r815u15n11 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7160(8165771) in scavenge on r816u35n11 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7161(8165772) in scavenge on r807u11n01 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7162(8165773) in scavenge on r914u05n01 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7163(8165774) in scavenge on r805u11n01 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7164(8165775) in scavenge on r805u14n01 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_958(7573127) in scavenge on r801u32n02 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_639(7571604) in scavenge on r912u32n02 Nov 05 06:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_84(7572251) in scavenge on r912u36n03 Nov 05 06:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6895(8165142) WEXITSTATUS 1 Nov 05 06:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6895(8165142) done Nov 05 06:48:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2465(8165776) in scavenge on r917u13n03 Nov 05 06:48:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2466(8165777) in scavenge on r917u29n01 Nov 05 06:48:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2467(8165778) in scavenge on r917u29n01 Nov 05 06:48:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2468(8165779) in scavenge on r917u29n02 Nov 05 06:48:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2469(8165780) in scavenge on r917u29n02 Nov 05 06:48:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2470(8165781) in scavenge on r815u15n09 Nov 05 06:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_501(8164066) WEXITSTATUS 0 Nov 05 06:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_501(8164066) done Nov 05 06:48:25 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2586(8127785) Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:48:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:48:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_55(8135754) WEXITSTATUS 0 Nov 05 06:48:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_55(8135754) done Nov 05 06:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_424(8163967) WEXITSTATUS 0 Nov 05 06:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_424(8163967) done Nov 05 06:48:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_602(8164246) WEXITSTATUS 0 Nov 05 06:48:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_602(8164246) done Nov 05 06:48:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7131(8165714) WEXITSTATUS 1 Nov 05 06:48:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7131(8165714) done Nov 05 06:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6406(8164179) WEXITSTATUS 1 Nov 05 06:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6406(8164179) done Nov 05 06:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3846(8165767) WEXITSTATUS 0 Nov 05 06:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3846(8165767) done Nov 05 06:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_504(8164069) WEXITSTATUS 0 Nov 05 06:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_504(8164069) done Nov 05 06:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_512(8164077) WEXITSTATUS 0 Nov 05 06:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_512(8164077) done Nov 05 06:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_540(8164121) WEXITSTATUS 0 Nov 05 06:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_540(8164121) done Nov 05 06:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6648(8164725) WEXITSTATUS 1 Nov 05 06:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6648(8164725) done Nov 05 06:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6881(8165111) WEXITSTATUS 1 Nov 05 06:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6881(8165111) done Nov 05 06:49:26 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:49:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1808(8158246) WEXITSTATUS 0 Nov 05 06:49:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1808(8158246) done Nov 05 06:49:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3113(8130966) WEXITSTATUS 1 Nov 05 06:49:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3113(8130966) done Nov 05 06:49:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_518(8164083) WEXITSTATUS 0 Nov 05 06:49:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_518(8164083) done Nov 05 06:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_606(8164250) WEXITSTATUS 0 Nov 05 06:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_606(8164250) done Nov 05 06:50:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6487(8164403) WEXITSTATUS 1 Nov 05 06:50:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6487(8164403) done Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:50:30 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:50:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3847(8165782) in gpu on r4513u29n01 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7165(8165783) in scavenge on r802u04n04 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7166(8165784) in scavenge on r802u09n03 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7167(8165785) in scavenge on r802u09n04 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7168(8165786) in scavenge on r815u03n09 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7169(8165787) in scavenge on r815u29n11 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7170(8165788) in scavenge on r805u04n02 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7171(8165789) in scavenge on r805u25n03 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7172(8165790) in scavenge on r913u01n03 Nov 05 06:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_188(7572356) in scavenge on r917u29n04 Nov 05 06:50:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2589(8127796) Nov 05 06:50:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2471(8165791) in scavenge on r906u13n01 Nov 05 06:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_452(8164005) WEXITSTATUS 0 Nov 05 06:50:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_452(8164005) done Nov 05 06:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6840(8165035) WEXITSTATUS 1 Nov 05 06:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6840(8165035) done Nov 05 06:50:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_590(8164209) WEXITSTATUS 0 Nov 05 06:50:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_590(8164209) done Nov 05 06:51:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_623(8164287) WEXITSTATUS 0 Nov 05 06:51:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_623(8164287) done Nov 05 06:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_556(8164152) WEXITSTATUS 0 Nov 05 06:51:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_556(8164152) done Nov 05 06:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1824(8158480) WEXITSTATUS 0 Nov 05 06:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1824(8158480) done Nov 05 06:51:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2608(8152875) WEXITSTATUS 0 Nov 05 06:51:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2608(8152875) done Nov 05 06:51:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7068(8165561) WEXITSTATUS 1 Nov 05 06:51:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7068(8165561) done Nov 05 06:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3553(8135863) WEXITSTATUS 1 Nov 05 06:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3553(8135863) done Nov 05 06:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6702(8164807) WEXITSTATUS 1 Nov 05 06:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6702(8164807) done Nov 05 06:51:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_616(8164280) WEXITSTATUS 0 Nov 05 06:51:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_616(8164280) done Nov 05 06:51:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2472(8165792) NodeList=r909u10n04 #CPUs=1 Partition=day Nov 05 06:51:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144145 NodeList=r801u23n01,r915u38n01 #CPUs=84 Partition=scavenge Nov 05 06:51:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3847(8165782) WEXITSTATUS 0 Nov 05 06:51:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3847(8165782) done Nov 05 06:51:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5771(8162713) WEXITSTATUS 1 Nov 05 06:51:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5771(8162713) done Nov 05 06:52:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6954(8165246) WEXITSTATUS 1 Nov 05 06:52:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6954(8165246) done Nov 05 06:52:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5869(8162841) WEXITSTATUS 1 Nov 05 06:52:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5869(8162841) done Nov 05 06:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_620(8164284) WEXITSTATUS 0 Nov 05 06:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_620(8164284) done Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:52:31 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:52:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2452(8152610) WEXITSTATUS 0 Nov 05 06:52:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2452(8152610) done Nov 05 06:52:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_569(8164165) WEXITSTATUS 0 Nov 05 06:52:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_569(8164165) done Nov 05 06:52:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154663 WEXITSTATUS 0 Nov 05 06:52:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154663 done Nov 05 06:52:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154717 NodeList=r816u09n04 #CPUs=16 Partition=day Nov 05 06:52:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_5(8158250) WEXITSTATUS 1 Nov 05 06:52:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_5(8158250) done Nov 05 06:52:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2611(8152878) WEXITSTATUS 0 Nov 05 06:52:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2611(8152878) done Nov 05 06:52:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3845(8165766) WEXITSTATUS 0 Nov 05 06:52:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3845(8165766) done Nov 05 06:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6586(8164626) WEXITSTATUS 1 Nov 05 06:53:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6586(8164626) done Nov 05 06:53:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3848(8165793) in gpu on r808u14n01 Nov 05 06:53:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3849(8165794) in gpu on r4513u29n01 Nov 05 06:53:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2473(8165795) in day on r919u09n04 Nov 05 06:53:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 06:53:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:53:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7173(8165796) in scavenge on r801u09n04 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7174(8165797) in scavenge on r802u02n03 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7175(8165798) in scavenge on r815u15n10 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7176(8165799) in scavenge on r816u35n12 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7177(8165800) in scavenge on r807u11n04 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7178(8165801) in scavenge on r913u05n01 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7179(8165802) in scavenge on r913u24n02 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7180(8165803) in scavenge on r909u07n04 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7181(8165804) in scavenge on r909u07n04 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_194(7572362) in scavenge on r918u05n01 Nov 05 06:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_313(7572481) in scavenge on r919u05n01 Nov 05 06:53:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_541(8164122) WEXITSTATUS 0 Nov 05 06:53:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_541(8164122) done Nov 05 06:53:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_582(8164201) WEXITSTATUS 0 Nov 05 06:53:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_582(8164201) done Nov 05 06:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1807(8158245) WEXITSTATUS 0 Nov 05 06:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1807(8158245) done Nov 05 06:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_568(8164164) WEXITSTATUS 0 Nov 05 06:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_568(8164164) done Nov 05 06:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820342 WEXITSTATUS 0 Nov 05 06:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820342 done Nov 05 06:53:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820373 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 06:54:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1694(8156716) WEXITSTATUS 0 Nov 05 06:54:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1694(8156716) done Nov 05 06:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6585(8164625) WEXITSTATUS 1 Nov 05 06:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6585(8164625) done Nov 05 06:54:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6847(8165047) WEXITSTATUS 1 Nov 05 06:54:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6847(8165047) done Nov 05 06:54:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_22(8160521) WEXITSTATUS 0 Nov 05 06:54:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_22(8160521) done Nov 05 06:54:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4905(8158055) WEXITSTATUS 1 Nov 05 06:54:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4905(8158055) done Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:54:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:54:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2474(8165805) NodeList=r919u32n01 #CPUs=1 Partition=day Nov 05 06:54:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7160(8165771) WEXITSTATUS 1 Nov 05 06:54:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7160(8165771) done Nov 05 06:54:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_611(8164255) WEXITSTATUS 0 Nov 05 06:54:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_611(8164255) done Nov 05 06:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1858(8159885) WEXITSTATUS 0 Nov 05 06:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1858(8159885) done Nov 05 06:54:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 06:54:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907380 WEXITSTATUS 0 Nov 05 06:54:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907380 done Nov 05 06:54:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_550(8164146) WEXITSTATUS 0 Nov 05 06:54:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_550(8164146) done Nov 05 06:54:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2471(8152629) WEXITSTATUS 0 Nov 05 06:54:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2471(8152629) done Nov 05 06:55:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6822(8165001) WEXITSTATUS 1 Nov 05 06:55:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6822(8165001) done Nov 05 06:55:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820340 WEXITSTATUS 0 Nov 05 06:55:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820340 done Nov 05 06:55:32 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820374 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 06:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6582(8164622) WEXITSTATUS 1 Nov 05 06:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6582(8164622) done Nov 05 06:55:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2475(8165806) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 06:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_487(8164052) WEXITSTATUS 0 Nov 05 06:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_487(8164052) done Nov 05 06:55:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_491(8164056) WEXITSTATUS 0 Nov 05 06:55:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_491(8164056) done Nov 05 06:55:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2595(8127813) Nov 05 06:55:55 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2597(8127815) Nov 05 06:55:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6987(8165352) WEXITSTATUS 1 Nov 05 06:55:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6987(8165352) done Nov 05 06:56:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907381 in gpu on r805u38n01 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7182(8165807) in scavenge on r802u06n04 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7183(8165808) in scavenge on r815u23n07 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7184(8165809) in scavenge on r816u09n10 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7185(8165810) in scavenge on r816u35n11 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7186(8165811) in scavenge on r806u09n02 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7187(8165812) in scavenge on r807u05n01 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7188(8165813) in scavenge on r909u10n04 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7189(8165814) in scavenge on r909u13n01 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7190(8165815) in scavenge on r909u29n03 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_340(7572508) in scavenge on r917u13n01 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_393(7572561) in scavenge on r917u13n02 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_584(7572753) in scavenge on r917u29n01 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3838(8165650) WEXITSTATUS 0 Nov 05 06:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3838(8165650) done Nov 05 06:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2476(8165816) in scavenge on r906u13n01 Nov 05 06:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_588(8164207) WEXITSTATUS 0 Nov 05 06:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_588(8164207) done Nov 05 06:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7185(8165810) WEXITSTATUS 1 Nov 05 06:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7185(8165810) done Nov 05 06:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1831(8158510) WEXITSTATUS 0 Nov 05 06:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1831(8158510) done Nov 05 06:56:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1668(8156301) WEXITSTATUS 0 Nov 05 06:56:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1668(8156301) done Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:56:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1809(8158278) WEXITSTATUS 0 Nov 05 06:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1809(8158278) done Nov 05 06:56:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2477(8165817) NodeList=r909u04n04 #CPUs=1 Partition=day Nov 05 06:56:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2478(8165818) NodeList=r919u36n01 #CPUs=1 Partition=day Nov 05 06:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164595 WEXITSTATUS 0 Nov 05 06:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164595 done Nov 05 06:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7041(8165475) WEXITSTATUS 1 Nov 05 06:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7041(8165475) done Nov 05 06:56:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1503(8142944) WEXITSTATUS 0 Nov 05 06:56:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1503(8142944) done Nov 05 06:57:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7147(8165742) WEXITSTATUS 1 Nov 05 06:57:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7147(8165742) done Nov 05 06:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820343 WEXITSTATUS 0 Nov 05 06:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820343 done Nov 05 06:57:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820375 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 06:57:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6476(8164383) WEXITSTATUS 1 Nov 05 06:57:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6476(8164383) done Nov 05 06:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3835(8165558) WEXITSTATUS 0 Nov 05 06:57:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3835(8165558) done Nov 05 06:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6700(8164805) WEXITSTATUS 1 Nov 05 06:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6700(8164805) done Nov 05 06:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7057(8165520) WEXITSTATUS 1 Nov 05 06:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7057(8165520) done Nov 05 06:57:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_619(8164283) WEXITSTATUS 0 Nov 05 06:57:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_619(8164283) done Nov 05 06:58:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7094(8165635) WEXITSTATUS 1 Nov 05 06:58:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7094(8165635) done Nov 05 06:58:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7033(8165458) WEXITSTATUS 1 Nov 05 06:58:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7033(8165458) done Nov 05 06:58:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5614(8161715) WEXITSTATUS 1 Nov 05 06:58:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5614(8161715) done Nov 05 06:58:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3826(8165366) WEXITSTATUS 0 Nov 05 06:58:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3826(8165366) done Nov 05 06:58:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_548(8164144) WEXITSTATUS 0 Nov 05 06:58:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_548(8164144) done Nov 05 06:58:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7149(8165754) WEXITSTATUS 1 Nov 05 06:58:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7149(8165754) done Nov 05 06:58:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154662 WEXITSTATUS 0 Nov 05 06:58:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154662 done Nov 05 06:58:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3752(8164408) WEXITSTATUS 0 Nov 05 06:58:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3752(8164408) done Nov 05 06:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_604(8164248) WEXITSTATUS 0 Nov 05 06:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_604(8164248) done Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 06:58:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 06:58:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159201 WEXITSTATUS 0 Nov 05 06:58:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159201 done Nov 05 06:58:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154719 NodeList=r816u15n05 #CPUs=16 Partition=day Nov 05 06:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7092(8165633) WEXITSTATUS 1 Nov 05 06:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7092(8165633) done Nov 05 06:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3850(8165819) in gpu on r903u34n01 Nov 05 06:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3851(8165820) in gpu on r808u08n01 Nov 05 06:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3852(8165821) in gpu on r808u08n01 Nov 05 06:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3853(8165822) in gpu on r4513u20n01 Nov 05 06:58:53 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 06:58:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 06:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_526(8164107) WEXITSTATUS 0 Nov 05 06:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_526(8164107) done Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7191(8165823) in scavenge on r916u01n01 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7192(8165824) in scavenge on r916u01n01 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7193(8165825) in scavenge on r917u13n02 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7194(8165826) in scavenge on r816u23n11 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7195(8165827) in scavenge on r816u23n12 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7196(8165828) in scavenge on r816u35n11 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7197(8165829) in scavenge on r4513u29n01 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7198(8165830) in scavenge on r806u23n01 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7199(8165831) in scavenge on r909u29n04 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7200(8165832) in scavenge on r914u01n01 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7201(8165833) in scavenge on r918u05n02 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_708(7572877) in scavenge on r917u29n02 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1017(7573186) in scavenge on r805u36n01 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_60(8114326) in scavenge on r805u36n01 Nov 05 06:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_147(8114413) in scavenge on r805u36n01 Nov 05 06:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_562(8164158) WEXITSTATUS 0 Nov 05 06:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_562(8164158) done Nov 05 06:59:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2479(8165834) in scavenge on r806u28n01 Nov 05 06:59:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2480(8165835) in scavenge on r906u13n01 Nov 05 06:59:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6256(8163497) WEXITSTATUS 1 Nov 05 06:59:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6256(8163497) done Nov 05 06:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5679(8162250) WEXITSTATUS 1 Nov 05 06:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5679(8162250) done Nov 05 06:59:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_15(8158176) WEXITSTATUS 1 Nov 05 06:59:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_15(8158176) done Nov 05 06:59:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1869(8161559) WEXITSTATUS 0 Nov 05 06:59:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1869(8161559) done Nov 05 06:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2481(8165836) NodeList=r815u09n05 #CPUs=1 Partition=day Nov 05 06:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_580(8164199) WEXITSTATUS 0 Nov 05 06:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_580(8164199) done Nov 05 07:00:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 07:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 07:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 07:00:02 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 07:00:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 07:00:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 07:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_17(8158178) WEXITSTATUS 1 Nov 05 07:00:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_17(8158178) done Nov 05 07:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3849(8165794) WEXITSTATUS 0 Nov 05 07:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3849(8165794) done Nov 05 07:00:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6524(8164462) WEXITSTATUS 1 Nov 05 07:00:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6524(8164462) done Nov 05 07:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3850(8165819) WEXITSTATUS 0 Nov 05 07:00:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3850(8165819) done Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:00:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3830(8165427) WEXITSTATUS 0 Nov 05 07:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3830(8165427) done Nov 05 07:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7181(8165804) WEXITSTATUS 1 Nov 05 07:00:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7181(8165804) done Nov 05 07:00:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 07:00:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 07:00:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2482(8165837) NodeList=r815u15n02 #CPUs=1 Partition=day Nov 05 07:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 07:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 07:01:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 07:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6110(8163148) WEXITSTATUS 1 Nov 05 07:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6110(8163148) done Nov 05 07:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6893(8165140) WEXITSTATUS 1 Nov 05 07:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6893(8165140) done Nov 05 07:01:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1702(8156847) WEXITSTATUS 0 Nov 05 07:01:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1702(8156847) done Nov 05 07:01:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3854(8165838) in gpu on r903u34n01 Nov 05 07:01:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3855(8165839) in gpu on r808u14n01 Nov 05 07:01:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3856(8165840) in gpu on r4513u29n01 Nov 05 07:01:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2483(8165841) in day on r918u36n01 Nov 05 07:01:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6359(8163903) WEXITSTATUS 1 Nov 05 07:01:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6359(8163903) done Nov 05 07:01:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7157(8165768) WEXITSTATUS 1 Nov 05 07:01:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7157(8165768) done Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7202(8165842) in scavenge on r802u04n03 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7203(8165843) in scavenge on r903u01n01 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7204(8165844) in scavenge on r806u06n01 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7205(8165845) in scavenge on r912u36n04 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7206(8165846) in scavenge on r909u07n04 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7207(8165847) in scavenge on r909u13n04 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7208(8165848) in scavenge on r918u05n02 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7209(8165849) in scavenge on r918u05n03 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_311(7572479) in scavenge on r815u15n02 Nov 05 07:01:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_432(7572600) in scavenge on r815u15n02 Nov 05 07:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3854(8165838) WEXITSTATUS 0 Nov 05 07:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3854(8165838) done Nov 05 07:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1871(8161561) WEXITSTATUS 0 Nov 05 07:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1871(8161561) done Nov 05 07:01:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165850 InitPrio=20041 usec=3625 Nov 05 07:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165688 WEXITSTATUS 0 Nov 05 07:01:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165688 done Nov 05 07:02:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3848(8165793) WEXITSTATUS 0 Nov 05 07:02:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3848(8165793) done Nov 05 07:02:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_56(8135755) WEXITSTATUS 0 Nov 05 07:02:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_56(8135755) done Nov 05 07:02:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6882(8165112) WEXITSTATUS 1 Nov 05 07:02:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6882(8165112) done Nov 05 07:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1868(8161558) WEXITSTATUS 0 Nov 05 07:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1868(8161558) done Nov 05 07:02:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820346 WEXITSTATUS 0 Nov 05 07:02:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820346 done Nov 05 07:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820376 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 07:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165850 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 #CPUs=288 Partition=day Nov 05 07:02:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2464(8152622) WEXITSTATUS 0 Nov 05 07:02:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2464(8152622) done Nov 05 07:02:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1882(8162558) WEXITSTATUS 0 Nov 05 07:02:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1882(8162558) done Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154666 WEXITSTATUS 0 Nov 05 07:02:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154666 done Nov 05 07:02:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_581(8164200) WEXITSTATUS 0 Nov 05 07:02:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_581(8164200) done Nov 05 07:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7006(8165400) WEXITSTATUS 1 Nov 05 07:02:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7006(8165400) done Nov 05 07:02:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6958(8165278) WEXITSTATUS 1 Nov 05 07:02:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6958(8165278) done Nov 05 07:02:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7169(8165787) WEXITSTATUS 1 Nov 05 07:02:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7169(8165787) done Nov 05 07:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6963(8165283) WEXITSTATUS 1 Nov 05 07:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6963(8165283) done Nov 05 07:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 07:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 07:03:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 07:03:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7188(8165813) WEXITSTATUS 1 Nov 05 07:03:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7188(8165813) done Nov 05 07:03:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2608(8127848) Nov 05 07:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_565(8164161) WEXITSTATUS 0 Nov 05 07:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_565(8164161) done Nov 05 07:03:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154720 NodeList=r919u36n03 #CPUs=16 Partition=day Nov 05 07:03:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2484(8165851) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 07:03:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133198 WEXITSTATUS 0 Nov 05 07:03:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133198 done Nov 05 07:03:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1854(8159761) WEXITSTATUS 0 Nov 05 07:03:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1854(8159761) done Nov 05 07:03:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7004(8165398) WEXITSTATUS 1 Nov 05 07:03:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7004(8165398) done Nov 05 07:03:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162673 WEXITSTATUS 0 Nov 05 07:03:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162673 done Nov 05 07:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6977(8165323) WEXITSTATUS 1 Nov 05 07:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6977(8165323) done Nov 05 07:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3857(8165852) in gpu on r808u14n01 Nov 05 07:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3858(8165853) in gpu on r903u34n01 Nov 05 07:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2485(8165854) in day on r914u01n02 Nov 05 07:04:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2486(8165855) in day on r815u03n02 Nov 05 07:04:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:04:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 07:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6930(8165213) WEXITSTATUS 1 Nov 05 07:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6930(8165213) done Nov 05 07:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7059(8165522) WEXITSTATUS 1 Nov 05 07:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7059(8165522) done Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7210(8165856) in scavenge on r802u04n02 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7211(8165857) in scavenge on r917u29n02 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7212(8165858) in scavenge on r815u29n11 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7213(8165859) in scavenge on r909u10n04 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7214(8165860) in scavenge on r914u05n02 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7215(8165861) in scavenge on r913u01n04 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7216(8165862) in scavenge on r913u09n04 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7217(8165863) in scavenge on r918u09n01 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7218(8165864) in scavenge on r919u01n04 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7219(8165865) in scavenge on r919u05n04 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7220(8165866) in scavenge on r919u09n01 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7221(8165867) in scavenge on r919u32n03 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_936(8116410) in scavenge on r917u29n03 Nov 05 07:04:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1065(7570727) in scavenge on r816u35n03 Nov 05 07:04:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 07:04:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2619(8152886) WEXITSTATUS 0 Nov 05 07:04:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2619(8152886) done Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:04:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3857(8165852) WEXITSTATUS 0 Nov 05 07:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3857(8165852) done Nov 05 07:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7197(8165829) WEXITSTATUS 1 Nov 05 07:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7197(8165829) done Nov 05 07:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3858(8165853) WEXITSTATUS 0 Nov 05 07:05:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3858(8165853) done Nov 05 07:05:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7173(8165796) WEXITSTATUS 1 Nov 05 07:05:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7173(8165796) done Nov 05 07:05:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820345 WEXITSTATUS 0 Nov 05 07:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820345 done Nov 05 07:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820377 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 07:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7066(8165559) WEXITSTATUS 1 Nov 05 07:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7066(8165559) done Nov 05 07:05:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7024(8165449) WEXITSTATUS 1 Nov 05 07:05:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7024(8165449) done Nov 05 07:05:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162431 WEXITSTATUS 0 Nov 05 07:05:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162431 done Nov 05 07:06:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7040(8165474) WEXITSTATUS 1 Nov 05 07:06:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7040(8165474) done Nov 05 07:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7191(8165823) WEXITSTATUS 1 Nov 05 07:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7191(8165823) done Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:06:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:06:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1505(8142946) WEXITSTATUS 0 Nov 05 07:06:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1505(8142946) done Nov 05 07:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6886(8165116) WEXITSTATUS 1 Nov 05 07:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6886(8165116) done Nov 05 07:07:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3859(8165868) in gpu on r808u14n01 Nov 05 07:07:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3860(8165869) in gpu on r903u34n01 Nov 05 07:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6909(8165164) WEXITSTATUS 1 Nov 05 07:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6909(8165164) done Nov 05 07:07:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7222(8165870) in scavenge on r802u02n01 Nov 05 07:07:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7223(8165871) in scavenge on r916u01n01 Nov 05 07:07:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7224(8165872) in scavenge on r801u09n04 Nov 05 07:07:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7225(8165873) in scavenge on r801u14n01 Nov 05 07:07:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7226(8165874) in scavenge on r801u32n02 Nov 05 07:07:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7227(8165875) in scavenge on r815u29n12 Nov 05 07:07:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7228(8165876) in scavenge on r4513u29n01 Nov 05 07:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2487(8165877) in scavenge on r816u23n11 Nov 05 07:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2488(8165878) in scavenge on r816u23n11 Nov 05 07:07:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2489(8165879) in scavenge on r816u23n11 Nov 05 07:07:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7223(8165871) WEXITSTATUS 1 Nov 05 07:07:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7223(8165871) done Nov 05 07:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_16(8158318) WEXITSTATUS 1 Nov 05 07:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_16(8158318) done Nov 05 07:07:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2490(8165880) NodeList=r807u11n02 #CPUs=1 Partition=day Nov 05 07:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_874(8127988) WEXITSTATUS 0 Nov 05 07:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_874(8127988) done Nov 05 07:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1879(8162356) WEXITSTATUS 0 Nov 05 07:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1879(8162356) done Nov 05 07:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6315(8163745) WEXITSTATUS 1 Nov 05 07:07:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6315(8163745) done Nov 05 07:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162430 WEXITSTATUS 0 Nov 05 07:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162430 done Nov 05 07:07:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6219(8163371) WEXITSTATUS 1 Nov 05 07:07:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6219(8163371) done Nov 05 07:08:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7207(8165847) WEXITSTATUS 1 Nov 05 07:08:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7207(8165847) done Nov 05 07:08:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6625(8164690) WEXITSTATUS 1 Nov 05 07:08:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6625(8164690) done Nov 05 07:08:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5381(8159274) WEXITSTATUS 1 Nov 05 07:08:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5381(8159274) done Nov 05 07:08:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2491(8165881) NodeList=r815u09n02 #CPUs=1 Partition=day Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:08:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1425(8135958) WEXITSTATUS 0 Nov 05 07:08:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1425(8135958) done Nov 05 07:08:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7206(8165846) WEXITSTATUS 1 Nov 05 07:08:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7206(8165846) done Nov 05 07:09:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_705(8114974) Nov 05 07:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820344 WEXITSTATUS 0 Nov 05 07:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820344 done Nov 05 07:09:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820378 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 07:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7209(8165849) WEXITSTATUS 1 Nov 05 07:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7209(8165849) done Nov 05 07:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7038(8165472) WEXITSTATUS 1 Nov 05 07:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7038(8165472) done Nov 05 07:09:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7229(8165882) in scavenge on r916u01n01 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7230(8165883) in scavenge on r802u04n04 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7231(8165884) in scavenge on r802u11n01 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7232(8165885) in scavenge on r815u23n09 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7233(8165886) in scavenge on r4513u20n01 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7234(8165887) in scavenge on r4513u20n01 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7235(8165888) in scavenge on r903u34n01 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7236(8165889) in scavenge on r807u11n02 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7237(8165890) in scavenge on r807u11n02 Nov 05 07:09:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_424(7572592) in scavenge on r816u35n09 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2492(8165891) in scavenge on r806u28n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:10:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:10:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6875(8165105) WEXITSTATUS 1 Nov 05 07:10:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6875(8165105) done Nov 05 07:10:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6972(8165318) WEXITSTATUS 1 Nov 05 07:10:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6972(8165318) done Nov 05 07:10:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162432 WEXITSTATUS 0 Nov 05 07:10:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162432 done Nov 05 07:10:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5404(8159385) WEXITSTATUS 1 Nov 05 07:10:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5404(8159385) done Nov 05 07:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7183(8165808) WEXITSTATUS 1 Nov 05 07:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7183(8165808) done Nov 05 07:10:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7237(8165890) WEXITSTATUS 1 Nov 05 07:10:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7237(8165890) done Nov 05 07:11:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1856(8159883) WEXITSTATUS 0 Nov 05 07:11:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1856(8159883) done Nov 05 07:11:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:11:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7065(8165528) WEXITSTATUS 1 Nov 05 07:11:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7065(8165528) done Nov 05 07:11:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7182(8165807) WEXITSTATUS 1 Nov 05 07:11:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7182(8165807) done Nov 05 07:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1506(8142947) WEXITSTATUS 0 Nov 05 07:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1506(8142947) done Nov 05 07:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3853(8165822) WEXITSTATUS 0 Nov 05 07:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3853(8165822) done Nov 05 07:11:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_364(8114631) Nov 05 07:11:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7126(8165709) WEXITSTATUS 1 Nov 05 07:11:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7126(8165709) done Nov 05 07:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6091(8163124) WEXITSTATUS 1 Nov 05 07:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6091(8163124) done Nov 05 07:11:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2493(8165892) NodeList=r918u32n04 #CPUs=1 Partition=day Nov 05 07:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6952(8165244) WEXITSTATUS 1 Nov 05 07:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6952(8165244) done Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:12:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7071(8165564) WEXITSTATUS 1 Nov 05 07:12:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7071(8165564) done Nov 05 07:12:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3861(8165893) in gpu on r4513u20n01 Nov 05 07:12:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165894 InitPrio=39669 usec=3717 Nov 05 07:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7112(8165691) WEXITSTATUS 1 Nov 05 07:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7112(8165691) done Nov 05 07:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1838(8158617) WEXITSTATUS 0 Nov 05 07:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1838(8158617) done Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7238(8165895) in scavenge on r801u06n01 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7239(8165896) in scavenge on r802u06n04 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7240(8165897) in scavenge on r904u01n03 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7241(8165898) in scavenge on r917u13n04 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7242(8165899) in scavenge on r815u23n07 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7243(8165900) in scavenge on r808u14n01 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7244(8165901) in scavenge on r807u11n02 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7245(8165902) in scavenge on r904u07n04 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7246(8165903) in scavenge on r909u07n04 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7247(8165904) in scavenge on r909u13n04 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7248(8165905) in scavenge on r918u05n03 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7249(8165906) in scavenge on r918u05n03 Nov 05 07:12:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_978(7571944) in scavenge on r802u06n02 Nov 05 07:12:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_486(8164051) WEXITSTATUS 0 Nov 05 07:12:43 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8164051: assoc 5369 TRES cpu grp_used_tres_run_secs underflow, tried to remove 115080 seconds when only 114720 remained. Nov 05 07:12:43 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8164051: assoc 5369 TRES cpu grp_used_tres_run_secs underflow, tried to remove 115080 seconds when only 114720 remained. Nov 05 07:12:43 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8164051: assoc 5369 TRES mem grp_used_tres_run_secs underflow, tried to remove 824893440 seconds when only 822312960 remained. Nov 05 07:12:43 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8164051: assoc 5369 TRES node grp_used_tres_run_secs underflow, tried to remove 115080 seconds when only 114720 remained. Nov 05 07:12:43 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8164051: assoc 5369 TRES mem grp_used_tres_run_secs underflow, tried to remove 824893440 seconds when only 822312960 remained. Nov 05 07:12:43 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8164051: assoc 5369 TRES node grp_used_tres_run_secs underflow, tried to remove 115080 seconds when only 114720 remained. Nov 05 07:12:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159119_486(8164051) done Nov 05 07:12:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165894 NodeList=r903u37n01 #CPUs=2 Partition=gpu_devel Nov 05 07:12:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7035(8165460) WEXITSTATUS 1 Nov 05 07:12:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7035(8165460) done Nov 05 07:13:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6660(8164739) WEXITSTATUS 1 Nov 05 07:13:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6660(8164739) done Nov 05 07:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3851(8165820) WEXITSTATUS 0 Nov 05 07:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3851(8165820) done Nov 05 07:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820347 WEXITSTATUS 0 Nov 05 07:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820347 done Nov 05 07:13:34 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820379 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 07:13:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7217(8165863) WEXITSTATUS 1 Nov 05 07:13:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7217(8165863) done Nov 05 07:13:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2058(8164513) WEXITSTATUS 0 Nov 05 07:13:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2058(8164513) done Nov 05 07:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_18(8158179) WEXITSTATUS 1 Nov 05 07:13:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_18(8158179) done Nov 05 07:13:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6992(8165357) WEXITSTATUS 1 Nov 05 07:13:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6992(8165357) done Nov 05 07:13:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7014(8165419) WEXITSTATUS 1 Nov 05 07:13:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7014(8165419) done Nov 05 07:13:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2494(8165907) NodeList=r815u15n03 #CPUs=1 Partition=day Nov 05 07:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3859(8165868) WEXITSTATUS 0 Nov 05 07:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3859(8165868) done Nov 05 07:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6619(8164679) WEXITSTATUS 1 Nov 05 07:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6619(8164679) done Nov 05 07:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1744(8157747) WEXITSTATUS 0 Nov 05 07:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1744(8157747) done Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:14:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_70(8135769) WEXITSTATUS 0 Nov 05 07:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_70(8135769) done Nov 05 07:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3861(8165893) WEXITSTATUS 0 Nov 05 07:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3861(8165893) done Nov 05 07:14:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6559(8164581) WEXITSTATUS 1 Nov 05 07:14:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6559(8164581) done Nov 05 07:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6554(8164507) WEXITSTATUS 1 Nov 05 07:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6554(8164507) done Nov 05 07:14:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2495(8165908) NodeList=r909u29n01 #CPUs=1 Partition=day Nov 05 07:15:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7026(8165451) WEXITSTATUS 1 Nov 05 07:15:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7026(8165451) done Nov 05 07:15:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3862(8165909) in gpu on r808u08n01 Nov 05 07:15:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3863(8165910) in gpu on r808u14n01 Nov 05 07:15:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3864(8165911) in gpu on r4513u20n01 Nov 05 07:15:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7250(8165912) in scavenge on r903u07n01 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7251(8165913) in scavenge on r903u07n01 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7252(8165914) in scavenge on r916u01n01 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7253(8165915) in scavenge on r805u02n01 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7254(8165916) in scavenge on r904u07n01 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7255(8165917) in scavenge on r909u07n01 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7256(8165918) in scavenge on r909u10n01 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7257(8165919) in scavenge on r909u13n02 Nov 05 07:15:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7258(8165920) in scavenge on r918u01n04 Nov 05 07:15:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6947(8165239) WEXITSTATUS 1 Nov 05 07:15:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6947(8165239) done Nov 05 07:15:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2496(8165921) in scavenge on r911u05n01 Nov 05 07:15:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2497(8165922) in scavenge on r815u23n07 Nov 05 07:15:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2498(8165923) in scavenge on r806u28n01 Nov 05 07:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4053(8147294) WEXITSTATUS 1 Nov 05 07:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4053(8147294) done Nov 05 07:15:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5676(8162168) WEXITSTATUS 1 Nov 05 07:15:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5676(8162168) done Nov 05 07:15:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7064(8165527) WEXITSTATUS 1 Nov 05 07:15:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7064(8165527) done Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:16:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:16:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3093(8130945) WEXITSTATUS 1 Nov 05 07:16:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3093(8130945) done Nov 05 07:16:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6907(8165162) WEXITSTATUS 1 Nov 05 07:16:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6907(8165162) done Nov 05 07:16:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2647(8127922) Nov 05 07:16:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6990(8165355) WEXITSTATUS 1 Nov 05 07:16:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6990(8165355) done Nov 05 07:16:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7230(8165883) WEXITSTATUS 1 Nov 05 07:16:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7230(8165883) done Nov 05 07:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5916(8162908) WEXITSTATUS 1 Nov 05 07:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5916(8162908) done Nov 05 07:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6913(8165168) WEXITSTATUS 1 Nov 05 07:16:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6913(8165168) done Nov 05 07:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2092(8164547) WEXITSTATUS 0 Nov 05 07:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2092(8164547) done Nov 05 07:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3855(8165839) WEXITSTATUS 0 Nov 05 07:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3855(8165839) done Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:18:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:18:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3865(8165924) in gpu on r808u14n01 Nov 05 07:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7201(8165833) WEXITSTATUS 1 Nov 05 07:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7201(8165833) done Nov 05 07:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5857(8162828) WEXITSTATUS 1 Nov 05 07:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5857(8162828) done Nov 05 07:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1873(8161755) WEXITSTATUS 0 Nov 05 07:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1873(8161755) done Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7259(8165925) in scavenge on r802u04n04 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7260(8165926) in scavenge on r802u09n03 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7261(8165927) in scavenge on r802u11n03 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7262(8165928) in scavenge on r815u15n09 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7263(8165929) in scavenge on r816u23n08 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7264(8165930) in scavenge on r805u02n01 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7265(8165931) in scavenge on r807u05n04 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7266(8165932) in scavenge on r913u01n02 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7267(8165933) in scavenge on r913u13n01 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7268(8165934) in scavenge on r918u05n03 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7269(8165935) in scavenge on r918u09n01 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7270(8165936) in scavenge on r918u32n04 Nov 05 07:18:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7271(8165937) in scavenge on r919u01n01 Nov 05 07:18:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2499(8165938) in scavenge on r808u08n01 Nov 05 07:19:03 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_754(7572923) Nov 05 07:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6936(8165219) WEXITSTATUS 1 Nov 05 07:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6936(8165219) done Nov 05 07:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162635 WEXITSTATUS 0 Nov 05 07:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162635 done Nov 05 07:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1736(8157686) WEXITSTATUS 0 Nov 05 07:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1736(8157686) done Nov 05 07:19:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2500(8165939) NodeList=r913u28n02 #CPUs=1 Partition=day Nov 05 07:19:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2501(8165940) NodeList=r919u32n03 #CPUs=1 Partition=day Nov 05 07:19:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6926(8165198) WEXITSTATUS 1 Nov 05 07:19:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6926(8165198) done Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:20:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_50(8135749) WEXITSTATUS 0 Nov 05 07:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_50(8135749) done Nov 05 07:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1832(8158511) WEXITSTATUS 0 Nov 05 07:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1832(8158511) done Nov 05 07:20:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2502(8165941) NodeList=r909u29n04 #CPUs=1 Partition=day Nov 05 07:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4643(8156247) WEXITSTATUS 1 Nov 05 07:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4643(8156247) done Nov 05 07:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:20:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7272(8165942) in scavenge on r815u03n11 Nov 05 07:20:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7273(8165943) in scavenge on r918u05n02 Nov 05 07:20:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7274(8165944) in scavenge on r913u01n02 Nov 05 07:20:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_439(7572607) in scavenge on r802u09n03 Nov 05 07:20:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_487(7572655) in scavenge on r815u23n01 Nov 05 07:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1813(8158359) WEXITSTATUS 0 Nov 05 07:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1813(8158359) done Nov 05 07:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7077(8165599) WEXITSTATUS 1 Nov 05 07:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7077(8165599) done Nov 05 07:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2070(8164525) WEXITSTATUS 0 Nov 05 07:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2070(8164525) done Nov 05 07:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7253(8165915) WEXITSTATUS 1 Nov 05 07:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7253(8165915) done Nov 05 07:21:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2503(8165945) NodeList=r914u01n01 #CPUs=1 Partition=day Nov 05 07:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7267(8165933) WEXITSTATUS 1 Nov 05 07:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7267(8165933) done Nov 05 07:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7069(8165562) WEXITSTATUS 1 Nov 05 07:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7069(8165562) done Nov 05 07:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7036(8165461) WEXITSTATUS 1 Nov 05 07:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7036(8165461) done Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:22:04 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2005(8164222) WEXITSTATUS 0 Nov 05 07:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2005(8164222) done Nov 05 07:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7242(8165899) WEXITSTATUS 1 Nov 05 07:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7242(8165899) done Nov 05 07:22:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7052(8165499) WEXITSTATUS 1 Nov 05 07:22:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7052(8165499) done Nov 05 07:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1872(8161726) WEXITSTATUS 0 Nov 05 07:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1872(8161726) done Nov 05 07:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3862(8165909) WEXITSTATUS 0 Nov 05 07:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3862(8165909) done Nov 05 07:22:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2504(8165946) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 07:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820348 WEXITSTATUS 0 Nov 05 07:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820348 done Nov 05 07:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7083(8165617) WEXITSTATUS 1 Nov 05 07:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7083(8165617) done Nov 05 07:22:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820380 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 07:23:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6730(8164845) WEXITSTATUS 1 Nov 05 07:23:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6730(8164845) done Nov 05 07:23:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3866(8165947) in gpu on r808u08n01 Nov 05 07:23:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7275(8165948) in scavenge on r802u09n02 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7276(8165949) in scavenge on r815u23n07 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7277(8165950) in scavenge on r4513u23n01 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7278(8165951) in scavenge on r807u08n01 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7279(8165952) in scavenge on r805u02n01 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7280(8165953) in scavenge on r805u09n02 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7281(8165954) in scavenge on r805u27n02 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7282(8165955) in scavenge on r912u36n03 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7283(8165956) in scavenge on r913u13n01 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135526 WEXITSTATUS 0 Nov 05 07:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135526 done Nov 05 07:23:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2505(8165957) in scavenge on r802u23n02 Nov 05 07:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5136(8158476) WEXITSTATUS 1 Nov 05 07:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5136(8158476) done Nov 05 07:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159202 WEXITSTATUS 0 Nov 05 07:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159202 done Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:24:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6837(8165032) WEXITSTATUS 1 Nov 05 07:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6837(8165032) done Nov 05 07:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7283(8165956) WEXITSTATUS 1 Nov 05 07:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7283(8165956) done Nov 05 07:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3866(8165947) WEXITSTATUS 0 Nov 05 07:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3866(8165947) done Nov 05 07:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154687 WEXITSTATUS 0 Nov 05 07:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154687 done Nov 05 07:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7009(8165414) WEXITSTATUS 1 Nov 05 07:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7009(8165414) done Nov 05 07:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5853(8162824) WEXITSTATUS 1 Nov 05 07:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5853(8162824) done Nov 05 07:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6645(8164722) WEXITSTATUS 1 Nov 05 07:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6645(8164722) done Nov 05 07:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6727(8164842) WEXITSTATUS 1 Nov 05 07:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6727(8164842) done Nov 05 07:24:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154721 NodeList=r815u29n04 #CPUs=16 Partition=day Nov 05 07:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1885(8162726) WEXITSTATUS 0 Nov 05 07:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1885(8162726) done Nov 05 07:24:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7245(8165902) WEXITSTATUS 1 Nov 05 07:24:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7245(8165902) done Nov 05 07:24:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820349 WEXITSTATUS 0 Nov 05 07:24:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820349 done Nov 05 07:24:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820381 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 07:25:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2506(8165958) NodeList=r815u23n04 #CPUs=1 Partition=day Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6503(8164432) WEXITSTATUS 1 Nov 05 07:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6503(8164432) done Nov 05 07:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154673 WEXITSTATUS 0 Nov 05 07:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154673 done Nov 05 07:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3867(8165959) in gpu on r808u08n01 Nov 05 07:26:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154724 in day on r816u03n04 Nov 05 07:26:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_10(8158279) WEXITSTATUS 1 Nov 05 07:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_10(8158279) done Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7284(8165960) in scavenge on r912u05n04 Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7285(8165961) in scavenge on r816u23n12 Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7286(8165962) in scavenge on r808u11n01 Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7287(8165963) in scavenge on r904u07n04 Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7288(8165964) in scavenge on r912u36n02 Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7289(8165965) in scavenge on r913u09n02 Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7290(8165966) in scavenge on r913u13n01 Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7291(8165967) in scavenge on r913u24n01 Nov 05 07:26:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7292(8165968) in scavenge on r913u28n02 Nov 05 07:26:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2507(8165969) in scavenge on r815u15n10 Nov 05 07:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7248(8165905) WEXITSTATUS 1 Nov 05 07:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7248(8165905) done Nov 05 07:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6610(8164670) WEXITSTATUS 1 Nov 05 07:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6610(8164670) done Nov 05 07:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2508(8165970) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 07:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6870(8165093) WEXITSTATUS 1 Nov 05 07:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6870(8165093) done Nov 05 07:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2097(8164552) WEXITSTATUS 0 Nov 05 07:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2097(8164552) done Nov 05 07:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6776(8164928) WEXITSTATUS 1 Nov 05 07:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6776(8164928) done Nov 05 07:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6677(8164763) WEXITSTATUS 1 Nov 05 07:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6677(8164763) done Nov 05 07:27:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7079(8165613) WEXITSTATUS 1 Nov 05 07:27:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7079(8165613) done Nov 05 07:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3865(8165924) WEXITSTATUS 0 Nov 05 07:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3865(8165924) done Nov 05 07:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7084(8165618) WEXITSTATUS 1 Nov 05 07:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7084(8165618) done Nov 05 07:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7080(8165614) WEXITSTATUS 1 Nov 05 07:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7080(8165614) done Nov 05 07:27:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:27:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 07:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162696 WEXITSTATUS 1 Nov 05 07:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162696 done Nov 05 07:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4629(8156221) WEXITSTATUS 1 Nov 05 07:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4629(8156221) done Nov 05 07:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154657 WEXITSTATUS 0 Nov 05 07:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154657 done Nov 05 07:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159203 WEXITSTATUS 0 Nov 05 07:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159203 done Nov 05 07:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4543(8155948) WEXITSTATUS 1 Nov 05 07:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4543(8155948) done Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:28:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:28:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_3(8158221) WEXITSTATUS 1 Nov 05 07:28:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_3(8158221) done Nov 05 07:28:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154722 NodeList=r806u23n01 #CPUs=16 Partition=day Nov 05 07:28:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2509(8165971) NodeList=r919u36n03 #CPUs=1 Partition=day Nov 05 07:28:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 07:28:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3868(8165972) in gpu on r808u14n01 Nov 05 07:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2238(8152030) WEXITSTATUS 0 Nov 05 07:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2238(8152030) done Nov 05 07:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2138(8164653) WEXITSTATUS 0 Nov 05 07:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2138(8164653) done Nov 05 07:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7258(8165920) WEXITSTATUS 1 Nov 05 07:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7258(8165920) done Nov 05 07:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7293(8165973) in scavenge on r908u28n02 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7294(8165974) in scavenge on r907u05n03 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7295(8165975) in scavenge on r801u04n01 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7296(8165976) in scavenge on r917u29n03 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7297(8165977) in scavenge on r815u09n12 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7298(8165978) in scavenge on r816u15n12 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7299(8165979) in scavenge on r805u25n01 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7300(8165980) in scavenge on r807u08n01 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7301(8165981) in scavenge on r918u05n03 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7302(8165982) in scavenge on r918u09n02 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7303(8165983) in scavenge on r815u03n03 Nov 05 07:29:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_507(7572675) in scavenge on r908u24n01 Nov 05 07:29:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1424(8135957) WEXITSTATUS 0 Nov 05 07:29:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1424(8135957) done Nov 05 07:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2343(8126316) WEXITSTATUS 1 Nov 05 07:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2343(8126316) done Nov 05 07:29:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1590(8152160) WEXITSTATUS 0 Nov 05 07:29:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1590(8152160) done Nov 05 07:29:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_71(7572238) Nov 05 07:29:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_760(7572929) Nov 05 07:29:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2510(8165984) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 07:29:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2511(8165985) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 07:29:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165986 InitPrio=20041 usec=4846 Nov 05 07:29:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165850 WEXITSTATUS 0 Nov 05 07:29:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165850 done Nov 05 07:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5930(8162925) WEXITSTATUS 1 Nov 05 07:29:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5930(8162925) done Nov 05 07:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1889(8162848) WEXITSTATUS 0 Nov 05 07:29:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1889(8162848) done Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:30:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:30:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7063(8165526) WEXITSTATUS 1 Nov 05 07:30:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7063(8165526) done Nov 05 07:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6370(8163998) WEXITSTATUS 1 Nov 05 07:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6370(8163998) done Nov 05 07:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6191(8163313) WEXITSTATUS 1 Nov 05 07:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6191(8163313) done Nov 05 07:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2100(8164555) WEXITSTATUS 0 Nov 05 07:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2100(8164555) done Nov 05 07:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 07:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 07:30:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165986 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 #CPUs=288 Partition=day Nov 05 07:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7272(8165942) WEXITSTATUS 1 Nov 05 07:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7272(8165942) done Nov 05 07:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 07:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 07:31:02 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 07:31:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6988(8165353) WEXITSTATUS 1 Nov 05 07:31:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6988(8165353) done Nov 05 07:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3856(8165840) WEXITSTATUS 0 Nov 05 07:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3856(8165840) done Nov 05 07:31:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3869(8165987) in gpu on r4513u29n01 Nov 05 07:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7269(8165935) WEXITSTATUS 1 Nov 05 07:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7269(8165935) done Nov 05 07:31:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7143(8165738) WEXITSTATUS 1 Nov 05 07:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7143(8165738) done Nov 05 07:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6771(8164920) WEXITSTATUS 1 Nov 05 07:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6771(8164920) done Nov 05 07:31:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3717(8163440) WEXITSTATUS 0 Nov 05 07:31:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3717(8163440) done Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7304(8165988) in scavenge on r802u32n01 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7305(8165989) in scavenge on r815u03n11 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7306(8165990) in scavenge on r815u15n11 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7307(8165991) in scavenge on r903u40n01 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7308(8165992) in scavenge on r805u06n02 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7309(8165993) in scavenge on r918u01n04 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7310(8165994) in scavenge on r918u05n04 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7311(8165995) in scavenge on r816u29n05 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7312(8165996) in scavenge on r815u03n08 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7313(8165997) in scavenge on r815u09n02 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1158(7570820) in scavenge on r802u04n02 Nov 05 07:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_41(7571005) in scavenge on r815u09n06 Nov 05 07:31:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2512(8165998) in scavenge on r816u23n08 Nov 05 07:31:53 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 07:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8165999 InitPrio=24102 usec=6060 Nov 05 07:31:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165999 NodeList=r802u23n02 #CPUs=1 Partition=pi_hodgson Nov 05 07:32:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154678 WEXITSTATUS 0 Nov 05 07:32:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154678 done Nov 05 07:32:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2665(8128001) Nov 05 07:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2131(8164644) WEXITSTATUS 0 Nov 05 07:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2131(8164644) done Nov 05 07:32:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8165999 uid 23592 Nov 05 07:32:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_62(8135761) WEXITSTATUS 0 Nov 05 07:32:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_62(8135761) done Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:32:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:32:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166000 InitPrio=24102 usec=4835 Nov 05 07:32:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166000 NodeList=r802u23n02 #CPUs=1 Partition=pi_hodgson Nov 05 07:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5363(8159173) WEXITSTATUS 1 Nov 05 07:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5363(8159173) done Nov 05 07:32:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154723 NodeList=r816u15n01 #CPUs=16 Partition=day Nov 05 07:32:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7304(8165988) WEXITSTATUS 1 Nov 05 07:32:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7304(8165988) done Nov 05 07:33:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 07:33:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 07:33:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 07:33:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5894(8162870) WEXITSTATUS 1 Nov 05 07:33:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5894(8162870) done Nov 05 07:33:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7280(8165953) WEXITSTATUS 1 Nov 05 07:33:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7280(8165953) done Nov 05 07:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820352 WEXITSTATUS 0 Nov 05 07:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820352 done Nov 05 07:33:31 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820382 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 07:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6763(8164899) WEXITSTATUS 1 Nov 05 07:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6763(8164899) done Nov 05 07:34:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7051(8165498) WEXITSTATUS 1 Nov 05 07:34:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7051(8165498) done Nov 05 07:34:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7243(8165900) WEXITSTATUS 1 Nov 05 07:34:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7243(8165900) done Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:34:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:34:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3870(8166001) in gpu on r903u34n01 Nov 05 07:34:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2098(8164553) WEXITSTATUS 0 Nov 05 07:34:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2098(8164553) done Nov 05 07:34:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154664 WEXITSTATUS 0 Nov 05 07:34:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154664 done Nov 05 07:34:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_6(7570969) Nov 05 07:34:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_82(7572249) Nov 05 07:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2067(8164522) WEXITSTATUS 0 Nov 05 07:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2067(8164522) done Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7314(8166002) in scavenge on r802u32n01 Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7315(8166003) in scavenge on r903u01n01 Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7316(8166004) in scavenge on r808u14n01 Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7317(8166005) in scavenge on r907u32n01 Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7318(8166006) in scavenge on r909u13n02 Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7319(8166007) in scavenge on r912u36n02 Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7320(8166008) in scavenge on r914u05n01 Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7321(8166009) in scavenge on r918u09n01 Nov 05 07:34:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_548(7571513) in scavenge on r805u09n02 Nov 05 07:34:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2513(8166010) in scavenge on r816u35n09 Nov 05 07:34:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2125(8164600) WEXITSTATUS 0 Nov 05 07:34:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2125(8164600) done Nov 05 07:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154725 NodeList=r912u28n01 #CPUs=16 Partition=day Nov 05 07:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2514(8166011) NodeList=r815u35n06 #CPUs=1 Partition=day Nov 05 07:35:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6474(8164381) WEXITSTATUS 1 Nov 05 07:35:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6474(8164381) done Nov 05 07:35:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6513(8164446) WEXITSTATUS 1 Nov 05 07:35:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6513(8164446) done Nov 05 07:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1532(8145779) WEXITSTATUS 0 Nov 05 07:35:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1532(8145779) done Nov 05 07:36:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7320(8166008) WEXITSTATUS 1 Nov 05 07:36:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7320(8166008) done Nov 05 07:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154703 WEXITSTATUS 0 Nov 05 07:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154703 done Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:36:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:36:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3870(8166001) WEXITSTATUS 0 Nov 05 07:36:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3870(8166001) done Nov 05 07:36:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7113(8165692) WEXITSTATUS 1 Nov 05 07:36:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7113(8165692) done Nov 05 07:36:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154726 NodeList=r815u35n04 #CPUs=16 Partition=day Nov 05 07:36:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2164(8164769) WEXITSTATUS 0 Nov 05 07:36:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2164(8164769) done Nov 05 07:36:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1501(8142528) WEXITSTATUS 0 Nov 05 07:36:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1501(8142528) done Nov 05 07:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7221(8165867) WEXITSTATUS 1 Nov 05 07:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7221(8165867) done Nov 05 07:37:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3871(8166012) in gpu on r903u34n01 Nov 05 07:37:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:37:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2551(8152818) WEXITSTATUS 0 Nov 05 07:37:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2551(8152818) done Nov 05 07:37:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7322(8166013) in scavenge on r911u01n01 Nov 05 07:37:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7323(8166014) in scavenge on r816u15n10 Nov 05 07:37:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7324(8166015) in scavenge on r914u05n01 Nov 05 07:37:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7325(8166016) in scavenge on r919u32n03 Nov 05 07:37:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7326(8166017) in scavenge on r805u09n04 Nov 05 07:37:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_480(7571445) in scavenge on r802u09n01 Nov 05 07:37:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2515(8166018) in scavenge on r906u28n01 Nov 05 07:37:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2516(8166019) in scavenge on r906u13n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2686(8153073) WEXITSTATUS 0 Nov 05 07:37:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2686(8153073) done Nov 05 07:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7203(8165843) WEXITSTATUS 1 Nov 05 07:37:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7203(8165843) done Nov 05 07:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159204 WEXITSTATUS 0 Nov 05 07:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159204 done Nov 05 07:38:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2042(8164456) WEXITSTATUS 0 Nov 05 07:38:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2042(8164456) done Nov 05 07:38:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2007(8164260) WEXITSTATUS 0 Nov 05 07:38:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2007(8164260) done Nov 05 07:38:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7300(8165980) WEXITSTATUS 1 Nov 05 07:38:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7300(8165980) done Nov 05 07:38:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165591 WEXITSTATUS 137 Nov 05 07:38:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165591 done Nov 05 07:38:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7101(8165657) WEXITSTATUS 1 Nov 05 07:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7101(8165657) done Nov 05 07:38:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2517(8166020) NodeList=r815u15n03 #CPUs=1 Partition=day Nov 05 07:38:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2518(8166021) NodeList=r816u15n07 #CPUs=1 Partition=day Nov 05 07:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7105(8165672) WEXITSTATUS 1 Nov 05 07:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7105(8165672) done Nov 05 07:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3863(8165910) WEXITSTATUS 0 Nov 05 07:38:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3863(8165910) done Nov 05 07:39:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2130(8164619) WEXITSTATUS 0 Nov 05 07:39:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2130(8164619) done Nov 05 07:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2644(8153031) WEXITSTATUS 0 Nov 05 07:39:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2644(8153031) done Nov 05 07:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6588(8164628) WEXITSTATUS 1 Nov 05 07:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6588(8164628) done Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:39:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:39:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2519(8166022) NodeList=r815u23n02 #CPUs=1 Partition=day Nov 05 07:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5536(8160871) WEXITSTATUS 1 Nov 05 07:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5536(8160871) done Nov 05 07:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6369(8163997) WEXITSTATUS 1 Nov 05 07:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6369(8163997) done Nov 05 07:39:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6978(8165324) WEXITSTATUS 1 Nov 05 07:39:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6978(8165324) done Nov 05 07:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7127(8165710) WEXITSTATUS 1 Nov 05 07:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7127(8165710) done Nov 05 07:39:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3872(8166023) in gpu on r808u14n01 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7327(8166024) in scavenge on r802u09n01 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7328(8166025) in scavenge on r903u01n01 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7329(8166026) in scavenge on r815u03n12 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7330(8166027) in scavenge on r816u29n11 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7331(8166028) in scavenge on r805u25n03 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7332(8166029) in scavenge on r806u04n04 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7333(8166030) in scavenge on r807u08n01 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7334(8166031) in scavenge on r913u24n03 Nov 05 07:39:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7335(8166032) in scavenge on r914u05n03 Nov 05 07:40:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_49(7572216) Nov 05 07:40:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_141(7572308) Nov 05 07:40:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_187(7572355) Nov 05 07:40:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2520(8166033) in scavenge on r815u09n11 Nov 05 07:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154660 WEXITSTATUS 0 Nov 05 07:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154660 done Nov 05 07:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7334(8166031) WEXITSTATUS 1 Nov 05 07:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7334(8166031) done Nov 05 07:40:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7219(8165865) WEXITSTATUS 1 Nov 05 07:40:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7219(8165865) done Nov 05 07:40:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2035(8164415) WEXITSTATUS 0 Nov 05 07:40:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2035(8164415) done Nov 05 07:40:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3872(8166023) WEXITSTATUS 0 Nov 05 07:40:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3872(8166023) done Nov 05 07:40:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_15(8158303) WEXITSTATUS 1 Nov 05 07:40:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_15(8158303) done Nov 05 07:40:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3832(8165443) WEXITSTATUS 0 Nov 05 07:40:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3832(8165443) done Nov 05 07:40:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154727 NodeList=r919u05n02 #CPUs=16 Partition=day Nov 05 07:40:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2521(8166034) NodeList=r918u32n03 #CPUs=1 Partition=day Nov 05 07:40:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2522(8166035) NodeList=r816u03n03 #CPUs=1 Partition=day Nov 05 07:40:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7309(8165993) WEXITSTATUS 1 Nov 05 07:40:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7309(8165993) done Nov 05 07:41:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2198(8164900) WEXITSTATUS 0 Nov 05 07:41:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2198(8164900) done Nov 05 07:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7291(8165967) WEXITSTATUS 1 Nov 05 07:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7291(8165967) done Nov 05 07:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6949(8165241) WEXITSTATUS 1 Nov 05 07:41:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6949(8165241) done Nov 05 07:41:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1855(8159882) WEXITSTATUS 0 Nov 05 07:41:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1855(8159882) done Nov 05 07:41:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2014(8164326) WEXITSTATUS 0 Nov 05 07:41:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2014(8164326) done Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:41:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:41:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2523(8166036) NodeList=r918u32n04 #CPUs=1 Partition=day Nov 05 07:41:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2524(8166037) NodeList=r815u15n04 #CPUs=1 Partition=day Nov 05 07:41:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2073(8164528) WEXITSTATUS 0 Nov 05 07:41:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2073(8164528) done Nov 05 07:41:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6998(8165392) WEXITSTATUS 1 Nov 05 07:41:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6998(8165392) done Nov 05 07:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7313(8165997) WEXITSTATUS 1 Nov 05 07:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7313(8165997) done Nov 05 07:42:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6427(8164234) WEXITSTATUS 1 Nov 05 07:42:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6427(8164234) done Nov 05 07:42:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6980(8165326) WEXITSTATUS 1 Nov 05 07:42:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6980(8165326) done Nov 05 07:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2071(8164526) WEXITSTATUS 0 Nov 05 07:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2071(8164526) done Nov 05 07:42:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_766(7572935) Nov 05 07:42:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_10(8114276) Nov 05 07:42:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3873(8166038) in gpu on r808u14n01 Nov 05 07:42:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3874(8166039) in gpu on r4513u23n01 Nov 05 07:42:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4534(8155939) WEXITSTATUS 1 Nov 05 07:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4534(8155939) done Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7336(8166040) in scavenge on r802u06n03 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7337(8166041) in scavenge on r903u26n01 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7338(8166042) in scavenge on r913u24n01 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7339(8166043) in scavenge on r913u24n03 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7340(8166044) in scavenge on r918u01n01 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7341(8166045) in scavenge on r918u01n03 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7342(8166046) in scavenge on r918u01n04 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7343(8166047) in scavenge on r918u32n03 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7344(8166048) in scavenge on r919u05n04 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_529(7572698) in scavenge on r802u04n04 Nov 05 07:42:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_265(8114531) in scavenge on r802u06n04 Nov 05 07:42:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2525(8166049) in scavenge on r815u15n10 Nov 05 07:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6285(8163604) WEXITSTATUS 1 Nov 05 07:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6285(8163604) done Nov 05 07:43:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3860(8165869) WEXITSTATUS 0 Nov 05 07:43:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3860(8165869) done Nov 05 07:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2165(8164770) WEXITSTATUS 0 Nov 05 07:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2165(8164770) done Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:43:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:43:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2069(8164524) WEXITSTATUS 0 Nov 05 07:43:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2069(8164524) done Nov 05 07:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7287(8165963) WEXITSTATUS 1 Nov 05 07:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7287(8165963) done Nov 05 07:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3868(8165972) WEXITSTATUS 0 Nov 05 07:43:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3868(8165972) done Nov 05 07:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2015(8164327) WEXITSTATUS 0 Nov 05 07:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2015(8164327) done Nov 05 07:43:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2526(8166050) NodeList=r815u23n06 #CPUs=1 Partition=day Nov 05 07:43:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1874(8161756) WEXITSTATUS 0 Nov 05 07:43:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1874(8161756) done Nov 05 07:43:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7296(8165976) WEXITSTATUS 1 Nov 05 07:43:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7296(8165976) done Nov 05 07:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7213(8165859) WEXITSTATUS 1 Nov 05 07:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7213(8165859) done Nov 05 07:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820350 WEXITSTATUS 0 Nov 05 07:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820350 done Nov 05 07:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2134(8164647) WEXITSTATUS 0 Nov 05 07:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2134(8164647) done Nov 05 07:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5029(8158325) WEXITSTATUS 1 Nov 05 07:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5029(8158325) done Nov 05 07:44:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820383 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 07:44:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:44:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3874(8166039) WEXITSTATUS 0 Nov 05 07:44:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3874(8166039) done Nov 05 07:44:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2435(8152479) WEXITSTATUS 0 Nov 05 07:44:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2435(8152479) done Nov 05 07:45:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2684(8128046) Nov 05 07:45:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6403(8164176) WEXITSTATUS 1 Nov 05 07:45:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6403(8164176) done Nov 05 07:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7307(8165991) WEXITSTATUS 1 Nov 05 07:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7307(8165991) done Nov 05 07:45:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3875(8166051) in gpu on r903u34n01 Nov 05 07:45:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3876(8166052) in gpu on r808u14n01 Nov 05 07:45:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3877(8166053) in gpu on r4513u23n01 Nov 05 07:45:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1841(8158620) WEXITSTATUS 0 Nov 05 07:45:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1841(8158620) done Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7345(8166054) in scavenge on r903u04n03 Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7346(8166055) in scavenge on r917u29n03 Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7347(8166056) in scavenge on r802u11n04 Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7348(8166057) in scavenge on r802u23n02 Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7349(8166058) in scavenge on r815u29n11 Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7350(8166059) in scavenge on r816u29n10 Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7351(8166060) in scavenge on r903u40n01 Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7352(8166061) in scavenge on r805u27n02 Nov 05 07:45:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_677(7572846) in scavenge on r802u11n04 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:45:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:45:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7117(8165696) WEXITSTATUS 1 Nov 05 07:45:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7117(8165696) done Nov 05 07:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2141(8164656) WEXITSTATUS 0 Nov 05 07:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2141(8164656) done Nov 05 07:45:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651306_2(7651311) Nov 05 07:45:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3877(8166053) WEXITSTATUS 0 Nov 05 07:45:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3877(8166053) done Nov 05 07:45:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2527(8166062) NodeList=r816u09n03 #CPUs=1 Partition=day Nov 05 07:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7121(8165700) WEXITSTATUS 1 Nov 05 07:45:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7121(8165700) done Nov 05 07:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1883(8162602) WEXITSTATUS 0 Nov 05 07:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1883(8162602) done Nov 05 07:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2126(8164601) WEXITSTATUS 0 Nov 05 07:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2126(8164601) done Nov 05 07:45:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1884(8162725) WEXITSTATUS 0 Nov 05 07:45:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1884(8162725) done Nov 05 07:45:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2133(8164646) WEXITSTATUS 0 Nov 05 07:45:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2133(8164646) done Nov 05 07:46:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2161(8164766) WEXITSTATUS 0 Nov 05 07:46:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2161(8164766) done Nov 05 07:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2074(8164529) WEXITSTATUS 0 Nov 05 07:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2074(8164529) done Nov 05 07:46:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3867(8165959) WEXITSTATUS 0 Nov 05 07:46:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3867(8165959) done Nov 05 07:46:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2528(8166063) NodeList=r815u09n05 #CPUs=1 Partition=day Nov 05 07:46:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2529(8166064) NodeList=r816u03n02 #CPUs=1 Partition=day Nov 05 07:46:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2530(8166065) NodeList=r816u03n07 #CPUs=1 Partition=day Nov 05 07:47:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2065(8164520) WEXITSTATUS 0 Nov 05 07:47:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2065(8164520) done Nov 05 07:47:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820351 WEXITSTATUS 0 Nov 05 07:47:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7820351 done Nov 05 07:47:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7820384 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:47:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7139(8165731) WEXITSTATUS 1 Nov 05 07:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7139(8165731) done Nov 05 07:47:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1118(7573287) Nov 05 07:47:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_135(8114401) Nov 05 07:47:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_459(8114726) Nov 05 07:47:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2687(8128049) Nov 05 07:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7054(8165501) WEXITSTATUS 1 Nov 05 07:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7054(8165501) done Nov 05 07:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1419(8135952) WEXITSTATUS 0 Nov 05 07:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1419(8135952) done Nov 05 07:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3875(8166051) WEXITSTATUS 0 Nov 05 07:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3875(8166051) done Nov 05 07:47:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7651306_2(7651311) NodeList=r815u35n07 #CPUs=1 Partition=scavenge_gpu Nov 05 07:47:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7115(8165694) WEXITSTATUS 1 Nov 05 07:47:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7115(8165694) done Nov 05 07:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6961(8165281) WEXITSTATUS 1 Nov 05 07:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6961(8165281) done Nov 05 07:47:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3878(8166066) in gpu on r903u34n01 Nov 05 07:47:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3879(8166067) in gpu on r808u08n01 Nov 05 07:47:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3880(8166068) in gpu on r4513u23n01 Nov 05 07:47:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:48:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7353(8166069) in scavenge on r917u13n02 Nov 05 07:48:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7354(8166070) in scavenge on r917u29n04 Nov 05 07:48:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7355(8166071) in scavenge on r815u15n12 Nov 05 07:48:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7356(8166072) in scavenge on r815u29n08 Nov 05 07:48:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7357(8166073) in scavenge on r815u29n12 Nov 05 07:48:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7358(8166074) in scavenge on r815u35n07 Nov 05 07:48:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7359(8166075) in scavenge on r805u06n04 Nov 05 07:48:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1028(7573197) in scavenge on r816u23n06 Nov 05 07:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2531(8166076) in scavenge on r815u15n10 Nov 05 07:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2532(8166077) in scavenge on r815u29n12 Nov 05 07:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2533(8166078) in scavenge on r816u29n08 Nov 05 07:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2534(8166079) in scavenge on r815u35n07 Nov 05 07:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2535(8166080) in scavenge on r815u35n07 Nov 05 07:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2536(8166081) in scavenge on r806u31n01 Nov 05 07:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7196(8165828) WEXITSTATUS 1 Nov 05 07:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7196(8165828) done Nov 05 07:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1896(8162998) WEXITSTATUS 0 Nov 05 07:48:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1896(8162998) done Nov 05 07:48:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2537(8166082) NodeList=r816u15n06 #CPUs=1 Partition=day Nov 05 07:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_14(8158299) WEXITSTATUS 1 Nov 05 07:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_14(8158299) done Nov 05 07:48:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2140(8164655) WEXITSTATUS 0 Nov 05 07:48:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2140(8164655) done Nov 05 07:48:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3879(8166067) WEXITSTATUS 0 Nov 05 07:48:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3879(8166067) done Nov 05 07:49:05 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:49:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2017(8164360) WEXITSTATUS 0 Nov 05 07:49:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2017(8164360) done Nov 05 07:49:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1881(8162519) WEXITSTATUS 0 Nov 05 07:49:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1881(8162519) done Nov 05 07:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154674 WEXITSTATUS 0 Nov 05 07:49:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154674 done Nov 05 07:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2066(8164521) WEXITSTATUS 0 Nov 05 07:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2066(8164521) done Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2006(8164223) WEXITSTATUS 0 Nov 05 07:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2006(8164223) done Nov 05 07:49:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2128(8164603) WEXITSTATUS 0 Nov 05 07:49:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2128(8164603) done Nov 05 07:49:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154728 NodeList=r913u09n02 #CPUs=16 Partition=day Nov 05 07:49:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2538(8166083) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 07:49:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2539(8166084) NodeList=r914u05n04 #CPUs=1 Partition=day Nov 05 07:49:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2540(8166085) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 07:49:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2541(8166086) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 07:49:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2542(8166087) NodeList=r816u15n01 #CPUs=1 Partition=day Nov 05 07:49:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2543(8166088) NodeList=r816u15n07 #CPUs=1 Partition=day Nov 05 07:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1880(8162428) WEXITSTATUS 0 Nov 05 07:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1880(8162428) done Nov 05 07:49:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_13(8158298) WEXITSTATUS 1 Nov 05 07:49:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_13(8158298) done Nov 05 07:50:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7331(8166028) WEXITSTATUS 1 Nov 05 07:50:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7331(8166028) done Nov 05 07:50:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2127(8164602) WEXITSTATUS 0 Nov 05 07:50:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2127(8164602) done Nov 05 07:50:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2123(8164587) WEXITSTATUS 0 Nov 05 07:50:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2123(8164587) done Nov 05 07:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2199(8164901) WEXITSTATUS 0 Nov 05 07:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2199(8164901) done Nov 05 07:50:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3881(8166089) in gpu on r808u08n01 Nov 05 07:50:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2544(8166090) in day on r919u36n02 Nov 05 07:50:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2545(8166091) in day on r816u03n03 Nov 05 07:50:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2546(8166092) in day on r816u03n08 Nov 05 07:50:42 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 07:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2094(8164549) WEXITSTATUS 0 Nov 05 07:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2094(8164549) done Nov 05 07:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7089(8165630) WEXITSTATUS 1 Nov 05 07:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7089(8165630) done Nov 05 07:50:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7360(8166093) in scavenge on r816u35n11 Nov 05 07:50:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7361(8166094) in scavenge on r805u25n03 Nov 05 07:50:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7362(8166095) in scavenge on r806u02n02 Nov 05 07:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2095(8164550) WEXITSTATUS 0 Nov 05 07:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2095(8164550) done Nov 05 07:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2029(8164398) WEXITSTATUS 0 Nov 05 07:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2029(8164398) done Nov 05 07:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2202(8164904) WEXITSTATUS 0 Nov 05 07:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2202(8164904) done Nov 05 07:51:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2028(8164397) WEXITSTATUS 0 Nov 05 07:51:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2028(8164397) done Nov 05 07:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2545(8152812) WEXITSTATUS 0 Nov 05 07:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2545(8152812) done Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:51:29 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:51:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2082(8164537) WEXITSTATUS 0 Nov 05 07:51:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2082(8164537) done Nov 05 07:51:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2547(8166096) NodeList=r815u23n01 #CPUs=1 Partition=day Nov 05 07:51:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2548(8166097) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 07:52:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6738(8164859) WEXITSTATUS 1 Nov 05 07:52:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6738(8164859) done Nov 05 07:52:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7299(8165979) WEXITSTATUS 1 Nov 05 07:52:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7299(8165979) done Nov 05 07:52:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154690 WEXITSTATUS 0 Nov 05 07:52:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154690 done Nov 05 07:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154669 WEXITSTATUS 0 Nov 05 07:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154669 done Nov 05 07:52:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154702 WEXITSTATUS 0 Nov 05 07:52:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154702 done Nov 05 07:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154729 NodeList=r909u07n03 #CPUs=16 Partition=day Nov 05 07:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154730 NodeList=r815u29n01 #CPUs=16 Partition=day Nov 05 07:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154731 NodeList=r815u29n01 #CPUs=16 Partition=day Nov 05 07:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2129(8164618) WEXITSTATUS 0 Nov 05 07:52:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2129(8164618) done Nov 05 07:52:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7263(8165929) WEXITSTATUS 1 Nov 05 07:52:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7263(8165929) done Nov 05 07:52:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6983(8165329) WEXITSTATUS 1 Nov 05 07:52:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6983(8165329) done Nov 05 07:53:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2101(8164556) WEXITSTATUS 0 Nov 05 07:53:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2101(8164556) done Nov 05 07:53:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1953(8163536) WEXITSTATUS 0 Nov 05 07:53:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1953(8163536) done Nov 05 07:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2030(8164399) WEXITSTATUS 0 Nov 05 07:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2030(8164399) done Nov 05 07:53:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2068(8164523) WEXITSTATUS 0 Nov 05 07:53:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2068(8164523) done Nov 05 07:53:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2549(8166098) in day on r815u03n06 Nov 05 07:53:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2550(8166099) in day on r815u03n06 Nov 05 07:53:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2551(8166100) in day on r815u35n02 Nov 05 07:53:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:53:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 07:53:27 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2268581793]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 05 07:53:27 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2268581793]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 05 07:53:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7363(8166101) in scavenge on r802u11n03 Nov 05 07:53:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7364(8166102) in scavenge on r917u13n01 Nov 05 07:53:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7365(8166103) in scavenge on r816u23n08 Nov 05 07:53:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7366(8166104) in scavenge on r805u25n01 Nov 05 07:53:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7346(8166055) WEXITSTATUS 1 Nov 05 07:53:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7346(8166055) done Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2022(8164365) WEXITSTATUS 0 Nov 05 07:53:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2022(8164365) done Nov 05 07:53:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2552(8166105) NodeList=r816u03n07 #CPUs=1 Partition=day Nov 05 07:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2072(8164527) WEXITSTATUS 0 Nov 05 07:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2072(8164527) done Nov 05 07:54:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158092 WEXITSTATUS 0 Nov 05 07:54:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158092 done Nov 05 07:54:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 07:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5688(8162310) WEXITSTATUS 1 Nov 05 07:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5688(8162310) done Nov 05 07:54:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154661 WEXITSTATUS 0 Nov 05 07:54:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154661 done Nov 05 07:54:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7194(8165826) WEXITSTATUS 1 Nov 05 07:54:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7194(8165826) done Nov 05 07:54:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154732 NodeList=r806u11n02 #CPUs=16 Partition=day Nov 05 07:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_49(8135748) WEXITSTATUS 0 Nov 05 07:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_49(8135748) done Nov 05 07:54:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7091(8165632) WEXITSTATUS 1 Nov 05 07:54:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7091(8165632) done Nov 05 07:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7361(8166094) WEXITSTATUS 1 Nov 05 07:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7361(8166094) done Nov 05 07:55:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7355(8166071) WEXITSTATUS 1 Nov 05 07:55:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7355(8166071) done Nov 05 07:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2040(8164441) WEXITSTATUS 0 Nov 05 07:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2040(8164441) done Nov 05 07:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2139(8164654) WEXITSTATUS 0 Nov 05 07:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2139(8164654) done Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:55:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:55:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2553(8166106) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 07:55:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2554(8166107) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 07:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6925(8165197) WEXITSTATUS 1 Nov 05 07:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6925(8165197) done Nov 05 07:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2148(8164707) WEXITSTATUS 0 Nov 05 07:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2148(8164707) done Nov 05 07:56:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2555(8166108) in day on r815u09n02 Nov 05 07:56:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2166(8164771) WEXITSTATUS 0 Nov 05 07:56:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2166(8164771) done Nov 05 07:56:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7367(8166109) in scavenge on r917u29n03 Nov 05 07:56:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7368(8166110) in scavenge on r802u30n01 Nov 05 07:56:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7369(8166111) in scavenge on r802u30n03 Nov 05 07:56:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7370(8166112) in scavenge on r815u15n12 Nov 05 07:56:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7371(8166113) in scavenge on r815u23n09 Nov 05 07:56:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7372(8166114) in scavenge on r816u23n11 Nov 05 07:56:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7373(8166115) in scavenge on r805u25n03 Nov 05 07:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1891(8162893) WEXITSTATUS 0 Nov 05 07:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1891(8162893) done Nov 05 07:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7212(8165858) WEXITSTATUS 1 Nov 05 07:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7212(8165858) done Nov 05 07:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1887(8162750) WEXITSTATUS 0 Nov 05 07:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1887(8162750) done Nov 05 07:56:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2556(8166116) NodeList=r815u09n02 #CPUs=1 Partition=day Nov 05 07:56:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2557(8166117) NodeList=r816u15n04 #CPUs=1 Partition=day Nov 05 07:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2135(8164648) WEXITSTATUS 0 Nov 05 07:56:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2135(8164648) done Nov 05 07:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7195(8165827) WEXITSTATUS 1 Nov 05 07:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7195(8165827) done Nov 05 07:57:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3871(8166012) WEXITSTATUS 0 Nov 05 07:57:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3871(8166012) done Nov 05 07:57:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2167(8164772) WEXITSTATUS 0 Nov 05 07:57:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2167(8164772) done Nov 05 07:57:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6923(8165183) WEXITSTATUS 1 Nov 05 07:57:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6923(8165183) done Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:57:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2151(8164710) WEXITSTATUS 0 Nov 05 07:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2151(8164710) done Nov 05 07:57:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166118 InitPrio=20041 usec=4331 Nov 05 07:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165986 WEXITSTATUS 0 Nov 05 07:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165986 done Nov 05 07:58:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166119 InitPrio=53336 usec=4426 Nov 05 07:58:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651306_5(7651458) Nov 05 07:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154686 WEXITSTATUS 0 Nov 05 07:58:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154686 done Nov 05 07:58:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7199(8165831) WEXITSTATUS 1 Nov 05 07:58:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7199(8165831) done Nov 05 07:58:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7251(8165913) WEXITSTATUS 1 Nov 05 07:58:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7251(8165913) done Nov 05 07:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7075(8165597) WEXITSTATUS 1 Nov 05 07:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7075(8165597) done Nov 05 07:58:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166118 in day on r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 Nov 05 07:58:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3882(8166120) in gpu on r4513u26n01 Nov 05 07:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_405(7572573) WEXITSTATUS 0 Nov 05 07:58:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_405(7572573) done Nov 05 07:58:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 07:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7374(8166121) in scavenge on r903u07n01 Nov 05 07:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7375(8166122) in scavenge on r917u13n02 Nov 05 07:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7376(8166123) in scavenge on r815u29n11 Nov 05 07:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7377(8166124) in scavenge on r816u23n12 Nov 05 07:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7378(8166125) in scavenge on r904u07n04 Nov 05 07:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7379(8166126) in scavenge on r912u36n02 Nov 05 07:58:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2558(8166127) in scavenge on r816u23n04 Nov 05 07:58:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2559(8166128) in scavenge on r816u35n04 Nov 05 07:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1878(8162256) WEXITSTATUS 0 Nov 05 07:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1878(8162256) done Nov 05 07:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7190(8165815) WEXITSTATUS 1 Nov 05 07:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7190(8165815) done Nov 05 07:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7125(8165708) WEXITSTATUS 1 Nov 05 07:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7125(8165708) done Nov 05 07:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3880(8166068) WEXITSTATUS 0 Nov 05 07:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3880(8166068) done Nov 05 07:59:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7357(8166073) WEXITSTATUS 1 Nov 05 07:59:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7357(8166073) done Nov 05 07:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7336(8166040) WEXITSTATUS 1 Nov 05 07:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7336(8166040) done Nov 05 07:59:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3882(8166120) WEXITSTATUS 0 Nov 05 07:59:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3882(8166120) done Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 07:59:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 07:59:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2122(8164586) WEXITSTATUS 0 Nov 05 07:59:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2122(8164586) done Nov 05 07:59:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7155(8165760) WEXITSTATUS 1 Nov 05 07:59:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7155(8165760) done Nov 05 07:59:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6031(8163047) WEXITSTATUS 1 Nov 05 07:59:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6031(8163047) done Nov 05 08:00:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907440 WEXITSTATUS 0 Nov 05 08:00:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907440 done Nov 05 08:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 08:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 08:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 08:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 08:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 08:00:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_74(8135773) WEXITSTATUS 0 Nov 05 08:00:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_74(8135773) done Nov 05 08:00:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6855(8165064) WEXITSTATUS 1 Nov 05 08:00:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6855(8165064) done Nov 05 08:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6931(8165214) WEXITSTATUS 1 Nov 05 08:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6931(8165214) done Nov 05 08:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_981(7571947) WEXITSTATUS 0 Nov 05 08:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_981(7571947) done Nov 05 08:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2184(8164851) WEXITSTATUS 0 Nov 05 08:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2184(8164851) done Nov 05 08:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7377(8166124) WEXITSTATUS 1 Nov 05 08:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7377(8166124) done Nov 05 08:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7088(8165629) WEXITSTATUS 1 Nov 05 08:01:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7088(8165629) done Nov 05 08:01:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r906u41n03 Nov 05 08:01:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r906u41n03 Nov 05 08:01:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3883(8166129) in gpu on r903u34n01 Nov 05 08:01:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3884(8166130) in gpu on r4513u23n01 Nov 05 08:01:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907441 in gpu on r805u38n01 Nov 05 08:01:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154735 in day on r815u09n07 Nov 05 08:01:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2560(8166131) in day on r815u15n02 Nov 05 08:01:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6995(8165389) WEXITSTATUS 1 Nov 05 08:01:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6995(8165389) done Nov 05 08:01:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7651306_5(7651458) in gpu on r4513u20n01 Nov 05 08:01:31 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2268611909]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 05 08:01:31 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2268611909]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 05 08:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7372(8166114) WEXITSTATUS 1 Nov 05 08:01:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7372(8166114) done Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7380(8166132) in scavenge on r802u06n03 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7381(8166133) in scavenge on r912u05n04 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7382(8166134) in scavenge on r904u01n03 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7383(8166135) in scavenge on r816u23n12 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7384(8166136) in scavenge on r815u23n08 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7385(8166137) in scavenge on r815u29n12 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7386(8166138) in scavenge on r805u23n01 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7387(8166139) in scavenge on r806u09n04 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7388(8166140) in scavenge on r904u04n02 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7389(8166141) in scavenge on r914u05n04 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7390(8166142) in scavenge on r918u32n04 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7391(8166143) in scavenge on r919u36n01 Nov 05 08:01:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1152(7573321) in scavenge on r815u35n07 Nov 05 08:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6943(8165226) WEXITSTATUS 1 Nov 05 08:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6943(8165226) done Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2099(8164554) WEXITSTATUS 0 Nov 05 08:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2099(8164554) done Nov 05 08:01:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 08:01:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 08:01:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 08:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6827(8165014) WEXITSTATUS 1 Nov 05 08:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6827(8165014) done Nov 05 08:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7151(8165756) WEXITSTATUS 1 Nov 05 08:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7151(8165756) done Nov 05 08:02:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166119 uid 26111 Nov 05 08:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7391(8166143) WEXITSTATUS 1 Nov 05 08:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7391(8166143) done Nov 05 08:02:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2084(8164539) WEXITSTATUS 0 Nov 05 08:02:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2084(8164539) done Nov 05 08:02:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1902(8163058) WEXITSTATUS 0 Nov 05 08:02:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1902(8163058) done Nov 05 08:02:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1901(8163041) WEXITSTATUS 0 Nov 05 08:02:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1901(8163041) done Nov 05 08:02:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2561(8166144) NodeList=r909u01n04 #CPUs=1 Partition=day Nov 05 08:02:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2562(8166145) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 08:02:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2563(8166146) NodeList=r816u15n07 #CPUs=1 Partition=day Nov 05 08:02:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7118(8165697) WEXITSTATUS 1 Nov 05 08:02:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7118(8165697) done Nov 05 08:02:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166147 InitPrio=53336 usec=5091 Nov 05 08:02:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6562(8164588) WEXITSTATUS 1 Nov 05 08:02:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6562(8164588) done Nov 05 08:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6412(8164185) WEXITSTATUS 1 Nov 05 08:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6412(8164185) done Nov 05 08:03:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2710(8128117) Nov 05 08:03:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1804(8158217) WEXITSTATUS 0 Nov 05 08:03:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1804(8158217) done Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:03:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:03:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 08:03:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 08:03:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 08:03:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2102(8164557) WEXITSTATUS 0 Nov 05 08:03:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2102(8164557) done Nov 05 08:04:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7070(8165563) WEXITSTATUS 1 Nov 05 08:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7070(8165563) done Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7392(8166148) in scavenge on r907u05n01 Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7393(8166149) in scavenge on r801u27n02 Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7394(8166150) in scavenge on r802u30n03 Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7395(8166151) in scavenge on r911u13n04 Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7396(8166152) in scavenge on r917u29n01 Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7397(8166153) in scavenge on r816u15n10 Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7398(8166154) in scavenge on r816u23n11 Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7399(8166155) in scavenge on r806u09n04 Nov 05 08:04:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7400(8166156) in scavenge on r918u09n04 Nov 05 08:04:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2564(8166157) in scavenge on r801u04n03 Nov 05 08:04:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2565(8166158) in scavenge on r904u13n01 Nov 05 08:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6507(8164436) WEXITSTATUS 1 Nov 05 08:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6507(8164436) done Nov 05 08:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2012(8164300) WEXITSTATUS 0 Nov 05 08:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2012(8164300) done Nov 05 08:04:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_912(8116386) WEXITSTATUS 0 Nov 05 08:04:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_912(8116386) done Nov 05 08:04:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2046(8164470) WEXITSTATUS 0 Nov 05 08:04:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2046(8164470) done Nov 05 08:04:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6911(8165166) WEXITSTATUS 1 Nov 05 08:04:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6911(8165166) done Nov 05 08:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2016(8164328) WEXITSTATUS 0 Nov 05 08:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2016(8164328) done Nov 05 08:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7385(8166137) WEXITSTATUS 1 Nov 05 08:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7385(8166137) done Nov 05 08:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2021(8164364) WEXITSTATUS 0 Nov 05 08:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2021(8164364) done Nov 05 08:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2132(8164645) WEXITSTATUS 0 Nov 05 08:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2132(8164645) done Nov 05 08:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6472(8164379) WEXITSTATUS 1 Nov 05 08:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6472(8164379) done Nov 05 08:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2050(8164484) WEXITSTATUS 0 Nov 05 08:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2050(8164484) done Nov 05 08:05:37 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166159 NodeList=(null) usec=32018 Nov 05 08:05:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_261(8114527) WEXITSTATUS 0 Nov 05 08:05:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_261(8114527) done Nov 05 08:05:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:05:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166159 NodeList=r904u13n01 #CPUs=8 Partition=pi_ying_rex Nov 05 08:05:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2433(8152477) WEXITSTATUS 0 Nov 05 08:05:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2433(8152477) done Nov 05 08:05:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166147 NodeList=r815u09n03 #CPUs=2 Partition=day Nov 05 08:05:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2566(8166160) NodeList=r815u15n03 #CPUs=1 Partition=day Nov 05 08:05:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2567(8166161) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 08:05:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2568(8166162) NodeList=r816u03n04 #CPUs=1 Partition=day Nov 05 08:05:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2569(8166163) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:05:46 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:05:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7363(8166101) WEXITSTATUS 1 Nov 05 08:05:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7363(8166101) done Nov 05 08:05:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_7(8158168) WEXITSTATUS 1 Nov 05 08:05:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_7(8158168) done Nov 05 08:06:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2150(8164709) WEXITSTATUS 0 Nov 05 08:06:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2150(8164709) done Nov 05 08:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1898(8163023) WEXITSTATUS 0 Nov 05 08:06:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1898(8163023) done Nov 05 08:06:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164701 WEXITSTATUS 0 Nov 05 08:06:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164701 done Nov 05 08:06:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2112(8164567) WEXITSTATUS 0 Nov 05 08:06:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2112(8164567) done Nov 05 08:06:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2045(8164469) WEXITSTATUS 0 Nov 05 08:06:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2045(8164469) done Nov 05 08:06:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6790(8164944) WEXITSTATUS 1 Nov 05 08:06:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6790(8164944) done Nov 05 08:06:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8165022 NodeList=r816u35n07,r907u34n01 #CPUs=8 Partition=gpu Nov 05 08:06:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2570(8166164) NodeList=r919u01n03 #CPUs=1 Partition=day Nov 05 08:06:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2571(8166165) NodeList=r815u09n07 #CPUs=1 Partition=day Nov 05 08:06:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2572(8166166) NodeList=r816u09n07 #CPUs=1 Partition=day Nov 05 08:06:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2573(8166167) NodeList=r816u09n08 #CPUs=1 Partition=day Nov 05 08:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6609(8164664) WEXITSTATUS 1 Nov 05 08:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6609(8164664) done Nov 05 08:06:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1890(8162851) WEXITSTATUS 0 Nov 05 08:06:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1890(8162851) done Nov 05 08:06:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158044 uid 25715 Nov 05 08:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165022 WEXITSTATUS 1 Nov 05 08:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165022 done Nov 05 08:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154667 WEXITSTATUS 0 Nov 05 08:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154667 done Nov 05 08:06:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2173(8164784) WEXITSTATUS 0 Nov 05 08:06:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2173(8164784) done Nov 05 08:06:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2034(8164414) WEXITSTATUS 0 Nov 05 08:06:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2034(8164414) done Nov 05 08:06:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154736 in day on r815u35n03 Nov 05 08:06:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2574(8166168) in day on r816u09n04 Nov 05 08:06:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2575(8166169) in day on r815u23n05 Nov 05 08:06:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2576(8166170) in day on r815u35n06 Nov 05 08:07:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6278(8163597) WEXITSTATUS 1 Nov 05 08:07:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6278(8163597) done Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7401(8166171) in scavenge on r801u04n03 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7402(8166172) in scavenge on r802u11n03 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7403(8166173) in scavenge on r915u32n01 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7404(8166174) in scavenge on r815u29n12 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7405(8166175) in scavenge on r907u34n01 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7406(8166176) in scavenge on r806u09n02 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7407(8166177) in scavenge on r906u41n04 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7408(8166178) in scavenge on r919u36n01 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1072(8115341) in scavenge on r912u28n04 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_940(7571906) in scavenge on r816u03n04 Nov 05 08:07:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_601(7571566) in scavenge on r805u38n01 Nov 05 08:07:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2149(8164708) WEXITSTATUS 0 Nov 05 08:07:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2149(8164708) done Nov 05 08:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_12(8158281) WEXITSTATUS 1 Nov 05 08:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_12(8158281) done Nov 05 08:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2168(8164773) WEXITSTATUS 0 Nov 05 08:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2168(8164773) done Nov 05 08:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6991(8165356) WEXITSTATUS 1 Nov 05 08:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6991(8165356) done Nov 05 08:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7402(8166172) WEXITSTATUS 1 Nov 05 08:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7402(8166172) done Nov 05 08:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_58(8135757) WEXITSTATUS 0 Nov 05 08:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_58(8135757) done Nov 05 08:07:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6950(8165242) WEXITSTATUS 1 Nov 05 08:07:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6950(8165242) done Nov 05 08:07:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6708(8164813) WEXITSTATUS 1 Nov 05 08:07:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6708(8164813) done Nov 05 08:07:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1877(8162255) WEXITSTATUS 0 Nov 05 08:07:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1877(8162255) done Nov 05 08:07:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2577(8166179) NodeList=r919u09n02 #CPUs=1 Partition=day Nov 05 08:07:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2578(8166180) NodeList=r815u09n05 #CPUs=1 Partition=day Nov 05 08:07:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2579(8166181) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6746(8164871) WEXITSTATUS 1 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6746(8164871) done Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3883(8166129) WEXITSTATUS 0 Nov 05 08:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3883(8166129) done Nov 05 08:07:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2117(8164573) WEXITSTATUS 0 Nov 05 08:07:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2117(8164573) done Nov 05 08:08:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6638(8164713) WEXITSTATUS 1 Nov 05 08:08:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6638(8164713) done Nov 05 08:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2083(8164538) WEXITSTATUS 0 Nov 05 08:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2083(8164538) done Nov 05 08:08:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7407(8166177) WEXITSTATUS 1 Nov 05 08:08:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7407(8166177) done Nov 05 08:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1894(8162958) WEXITSTATUS 0 Nov 05 08:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1894(8162958) done Nov 05 08:08:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2580(8166182) NodeList=r909u07n01 #CPUs=1 Partition=day Nov 05 08:08:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2581(8166183) NodeList=r815u23n04 #CPUs=1 Partition=day Nov 05 08:08:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2201(8164903) WEXITSTATUS 0 Nov 05 08:08:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2201(8164903) done Nov 05 08:08:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_17(8158319) WEXITSTATUS 1 Nov 05 08:08:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_17(8158319) done Nov 05 08:08:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7133(8165725) WEXITSTATUS 1 Nov 05 08:08:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7133(8165725) done Nov 05 08:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7324(8166015) WEXITSTATUS 1 Nov 05 08:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7324(8166015) done Nov 05 08:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_19(8158321) WEXITSTATUS 1 Nov 05 08:09:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_19(8158321) done Nov 05 08:09:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3881(8166089) WEXITSTATUS 0 Nov 05 08:09:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3881(8166089) done Nov 05 08:09:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7021(8165438) WEXITSTATUS 1 Nov 05 08:09:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7021(8165438) done Nov 05 08:09:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3885(8166184) in gpu on r903u34n01 Nov 05 08:09:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3886(8166185) in gpu on r808u08n01 Nov 05 08:09:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2582(8166186) in day on r918u05n04 Nov 05 08:09:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2583(8166187) in day on r919u09n02 Nov 05 08:09:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2232(8165005) WEXITSTATUS 0 Nov 05 08:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2232(8165005) done Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7409(8166188) in scavenge on r802u04n03 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7410(8166189) in scavenge on r802u11n03 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7411(8166190) in scavenge on r815u23n12 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7412(8166191) in scavenge on r816u29n10 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7413(8166192) in scavenge on r805u11n02 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7414(8166193) in scavenge on r906u41n04 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7415(8166194) in scavenge on r918u01n01 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7416(8166195) in scavenge on r919u09n02 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7417(8166196) in scavenge on r919u09n02 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7418(8166197) in scavenge on r909u10n04 Nov 05 08:09:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_628(7572797) in scavenge on r912u28n02 Nov 05 08:09:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2155(8164742) WEXITSTATUS 0 Nov 05 08:09:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2155(8164742) done Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:09:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:09:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_77(8135776) WEXITSTATUS 0 Nov 05 08:09:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_77(8135776) done Nov 05 08:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_6(8158251) WEXITSTATUS 1 Nov 05 08:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_6(8158251) done Nov 05 08:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6811(8164983) WEXITSTATUS 1 Nov 05 08:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6811(8164983) done Nov 05 08:10:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7179(8165802) WEXITSTATUS 1 Nov 05 08:10:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7179(8165802) done Nov 05 08:10:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7016(8165433) WEXITSTATUS 1 Nov 05 08:10:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7016(8165433) done Nov 05 08:10:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7238(8165895) WEXITSTATUS 1 Nov 05 08:10:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7238(8165895) done Nov 05 08:10:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2185(8164852) WEXITSTATUS 0 Nov 05 08:10:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2185(8164852) done Nov 05 08:10:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5355(8159089) WEXITSTATUS 1 Nov 05 08:10:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5355(8159089) done Nov 05 08:10:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6973(8165319) WEXITSTATUS 1 Nov 05 08:10:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6973(8165319) done Nov 05 08:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6338(8163841) WEXITSTATUS 1 Nov 05 08:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6338(8163841) done Nov 05 08:10:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_247(8114513) Nov 05 08:10:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2724(8128148) Nov 05 08:10:41 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2726(8128150) Nov 05 08:10:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:10:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2584(8166198) NodeList=r909u01n04 #CPUs=1 Partition=day Nov 05 08:10:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2585(8166199) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 08:10:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2586(8166200) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 08:10:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2158(8164745) WEXITSTATUS 0 Nov 05 08:10:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2158(8164745) done Nov 05 08:11:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2159(8164746) WEXITSTATUS 0 Nov 05 08:11:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2159(8164746) done Nov 05 08:11:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7215(8165861) WEXITSTATUS 1 Nov 05 08:11:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7215(8165861) done Nov 05 08:11:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6572(8164609) WEXITSTATUS 1 Nov 05 08:11:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6572(8164609) done Nov 05 08:11:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2203(8164905) WEXITSTATUS 0 Nov 05 08:11:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2203(8164905) done Nov 05 08:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2154(8164741) WEXITSTATUS 0 Nov 05 08:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2154(8164741) done Nov 05 08:11:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166201 InitPrio=57489 usec=4078 Nov 05 08:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1888(8162812) WEXITSTATUS 0 Nov 05 08:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1888(8162812) done Nov 05 08:11:34 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166201 NodeList=r801u30n02 #CPUs=2 Partition=pi_lora Nov 05 08:11:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2152(8164719) WEXITSTATUS 0 Nov 05 08:11:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2152(8164719) done Nov 05 08:11:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2587(8166202) NodeList=r816u09n03 #CPUs=1 Partition=day Nov 05 08:11:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2588(8166203) NodeList=r815u03n05 #CPUs=1 Partition=day Nov 05 08:11:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2589(8166204) NodeList=r815u09n01 #CPUs=1 Partition=day Nov 05 08:11:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2590(8166205) NodeList=r815u09n08 #CPUs=1 Partition=day Nov 05 08:11:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2591(8166206) NodeList=r815u15n02 #CPUs=1 Partition=day Nov 05 08:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6733(8164848) WEXITSTATUS 1 Nov 05 08:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6733(8164848) done Nov 05 08:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1569(8149614) WEXITSTATUS 0 Nov 05 08:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1569(8149614) done Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2206(8164908) WEXITSTATUS 0 Nov 05 08:11:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2206(8164908) done Nov 05 08:11:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7374(8166121) WEXITSTATUS 1 Nov 05 08:11:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7374(8166121) done Nov 05 08:12:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2592(8166207) in day on r816u03n01 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7419(8166208) in scavenge on r907u01n02 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7420(8166209) in scavenge on r801u06n01 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7421(8166210) in scavenge on r802u02n03 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7422(8166211) in scavenge on r903u07n01 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7423(8166212) in scavenge on r815u15n12 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7424(8166213) in scavenge on r816u29n11 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7425(8166214) in scavenge on r806u06n03 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7426(8166215) in scavenge on r909u01n04 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7427(8166216) in scavenge on r909u01n04 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7428(8166217) in scavenge on r909u13n01 Nov 05 08:12:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7429(8166218) in scavenge on r909u29n03 Nov 05 08:12:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7430(8166219) in scavenge on r909u29n04 Nov 05 08:12:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7431(8166220) in scavenge on r912u28n01 Nov 05 08:12:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2080(8164535) WEXITSTATUS 0 Nov 05 08:12:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2080(8164535) done Nov 05 08:12:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2079(8164534) WEXITSTATUS 0 Nov 05 08:12:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2079(8164534) done Nov 05 08:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2078(8164533) WEXITSTATUS 0 Nov 05 08:12:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2078(8164533) done Nov 05 08:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2038(8164428) WEXITSTATUS 0 Nov 05 08:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2038(8164428) done Nov 05 08:12:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2593(8166221) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 08:12:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7415(8166194) WEXITSTATUS 1 Nov 05 08:12:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7415(8166194) done Nov 05 08:12:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7172(8165790) WEXITSTATUS 1 Nov 05 08:12:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7172(8165790) done Nov 05 08:12:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7398(8166154) WEXITSTATUS 1 Nov 05 08:12:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7398(8166154) done Nov 05 08:13:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7176(8165799) WEXITSTATUS 1 Nov 05 08:13:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7176(8165799) done Nov 05 08:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1919(8163253) WEXITSTATUS 0 Nov 05 08:13:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1919(8163253) done Nov 05 08:13:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2081(8164536) WEXITSTATUS 0 Nov 05 08:13:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2081(8164536) done Nov 05 08:13:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_847(8115116) Nov 05 08:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144137 WEXITSTATUS 0 Nov 05 08:13:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144137 done Nov 05 08:13:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2594(8166222) NodeList=r816u15n02 #CPUs=1 Partition=day Nov 05 08:13:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144142 NodeList=r915u38n[03-04] #CPUs=96 Partition=scavenge Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:13:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2103(8164558) WEXITSTATUS 0 Nov 05 08:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2103(8164558) done Nov 05 08:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2145(8164687) WEXITSTATUS 0 Nov 05 08:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2145(8164687) done Nov 05 08:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7074(8165567) WEXITSTATUS 1 Nov 05 08:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7074(8165567) done Nov 05 08:14:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2595(8166223) NodeList=r816u15n05 #CPUs=1 Partition=day Nov 05 08:14:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166224 InitPrio=57489 usec=4007 Nov 05 08:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7382(8166134) WEXITSTATUS 1 Nov 05 08:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7382(8166134) done Nov 05 08:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5443(8159658) WEXITSTATUS 1 Nov 05 08:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5443(8159658) done Nov 05 08:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144142 WEXITSTATUS 0 Nov 05 08:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144142 done Nov 05 08:14:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166224 NodeList=r915u38n03 #CPUs=15 Partition=pi_lora Nov 05 08:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7394(8166150) WEXITSTATUS 1 Nov 05 08:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7394(8166150) done Nov 05 08:15:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:15:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2051(8164485) WEXITSTATUS 0 Nov 05 08:15:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2051(8164485) done Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7432(8166225) in scavenge on r802u30n03 Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7433(8166226) in scavenge on r904u01n03 Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7434(8166227) in scavenge on r816u23n11 Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7435(8166228) in scavenge on r816u35n12 Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7436(8166229) in scavenge on r903u24n01 Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7437(8166230) in scavenge on r904u04n01 Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7438(8166231) in scavenge on r918u01n01 Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7439(8166232) in scavenge on r913u01n03 Nov 05 08:15:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_21(7570984) in scavenge on r802u02n01 Nov 05 08:15:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6521(8164459) WEXITSTATUS 1 Nov 05 08:15:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6521(8164459) done Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2596(8166233) in scavenge on r815u29n08 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2597(8166234) in scavenge on r815u03n11 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2598(8166235) in scavenge on r815u15n09 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2599(8166236) in scavenge on r816u29n07 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2600(8166237) in scavenge on r816u29n07 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2601(8166238) in scavenge on r816u35n10 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2602(8166239) in scavenge on r816u35n10 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2603(8166240) in scavenge on r816u35n10 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2604(8166241) in scavenge on r816u35n10 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2605(8166242) in scavenge on r816u35n12 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2606(8166243) in scavenge on r816u35n12 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2607(8166244) in scavenge on r816u35n12 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2608(8166245) in scavenge on r912u09n01 Nov 05 08:15:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2609(8166246) in scavenge on r906u32n01 Nov 05 08:15:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2238(8165012) WEXITSTATUS 0 Nov 05 08:15:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2238(8165012) done Nov 05 08:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2019(8164362) WEXITSTATUS 0 Nov 05 08:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2019(8164362) done Nov 05 08:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154682 WEXITSTATUS 0 Nov 05 08:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154682 done Nov 05 08:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3876(8166052) WEXITSTATUS 0 Nov 05 08:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3876(8166052) done Nov 05 08:15:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154733 NodeList=r909u04n02 #CPUs=16 Partition=day Nov 05 08:15:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2610(8166247) NodeList=r919u01n04 #CPUs=1 Partition=day Nov 05 08:15:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2611(8166248) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 08:15:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2612(8166249) NodeList=r815u29n06 #CPUs=1 Partition=day Nov 05 08:15:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144143 NodeList=r907u36n01,r915u38n04 #CPUs=84 Partition=scavenge Nov 05 08:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2104(8164559) WEXITSTATUS 0 Nov 05 08:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2104(8164559) done Nov 05 08:15:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1886(8162727) WEXITSTATUS 0 Nov 05 08:15:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1886(8162727) done Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:15:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7232(8165885) WEXITSTATUS 1 Nov 05 08:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7232(8165885) done Nov 05 08:16:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1840(8158619) WEXITSTATUS 0 Nov 05 08:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1840(8158619) done Nov 05 08:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2245(8165037) WEXITSTATUS 0 Nov 05 08:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2245(8165037) done Nov 05 08:16:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2613(8166250) NodeList=r816u09n06 #CPUs=1 Partition=day Nov 05 08:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6428(8164235) WEXITSTATUS 1 Nov 05 08:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6428(8164235) done Nov 05 08:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2229(8164992) WEXITSTATUS 0 Nov 05 08:17:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2229(8164992) done Nov 05 08:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2047(8164471) WEXITSTATUS 0 Nov 05 08:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2047(8164471) done Nov 05 08:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2207(8164909) WEXITSTATUS 0 Nov 05 08:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2207(8164909) done Nov 05 08:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7175(8165798) WEXITSTATUS 1 Nov 05 08:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7175(8165798) done Nov 05 08:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159205 WEXITSTATUS 0 Nov 05 08:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159205 done Nov 05 08:17:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3887(8166251) in gpu on r808u14n01 Nov 05 08:17:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2614(8166252) in day on r815u23n06 Nov 05 08:17:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2615(8166253) in day on r815u15n05 Nov 05 08:17:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2616(8166254) in day on r815u15n07 Nov 05 08:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7440(8166255) in scavenge on r815u15n10 Nov 05 08:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7441(8166256) in scavenge on r815u23n09 Nov 05 08:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7442(8166257) in scavenge on r918u36n01 Nov 05 08:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7443(8166258) in scavenge on r913u01n04 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:17:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7039(8165473) WEXITSTATUS 1 Nov 05 08:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7039(8165473) done Nov 05 08:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2027(8164396) WEXITSTATUS 0 Nov 05 08:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2027(8164396) done Nov 05 08:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7264(8165930) WEXITSTATUS 1 Nov 05 08:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7264(8165930) done Nov 05 08:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2090(8164545) WEXITSTATUS 0 Nov 05 08:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2090(8164545) done Nov 05 08:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2142(8164668) WEXITSTATUS 0 Nov 05 08:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2142(8164668) done Nov 05 08:18:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2617(8166259) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 08:18:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2618(8166260) NodeList=r816u15n01 #CPUs=1 Partition=day Nov 05 08:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2170(8164775) WEXITSTATUS 0 Nov 05 08:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2170(8164775) done Nov 05 08:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3743(8164258) WEXITSTATUS 0 Nov 05 08:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3743(8164258) done Nov 05 08:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2156(8164743) WEXITSTATUS 0 Nov 05 08:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2156(8164743) done Nov 05 08:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7220(8165866) WEXITSTATUS 1 Nov 05 08:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7220(8165866) done Nov 05 08:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7225(8165873) WEXITSTATUS 1 Nov 05 08:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7225(8165873) done Nov 05 08:19:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166261 InitPrio=28336 usec=4633 Nov 05 08:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2639(8153026) WEXITSTATUS 0 Nov 05 08:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2639(8153026) done Nov 05 08:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2009(8164297) WEXITSTATUS 0 Nov 05 08:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2009(8164297) done Nov 05 08:19:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166261 uid 25715 Nov 05 08:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7433(8166226) WEXITSTATUS 1 Nov 05 08:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7433(8166226) done Nov 05 08:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2147(8164706) WEXITSTATUS 0 Nov 05 08:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2147(8164706) done Nov 05 08:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7380(8166132) WEXITSTATUS 1 Nov 05 08:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7380(8166132) done Nov 05 08:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2180(8164820) WEXITSTATUS 0 Nov 05 08:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2180(8164820) done Nov 05 08:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6910(8165165) WEXITSTATUS 1 Nov 05 08:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6910(8165165) done Nov 05 08:19:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166262 InitPrio=28336 usec=4573 Nov 05 08:19:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2619(8166263) NodeList=r918u05n04 #CPUs=1 Partition=day Nov 05 08:19:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2620(8166264) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 08:19:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2621(8166265) NodeList=r816u03n08 #CPUs=1 Partition=day Nov 05 08:19:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2622(8166266) NodeList=r815u03n03 #CPUs=1 Partition=day Nov 05 08:19:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2623(8166267) NodeList=r815u09n04 #CPUs=1 Partition=day Nov 05 08:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159111 WEXITSTATUS 0 Nov 05 08:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159111 done Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:19:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6516(8164451) WEXITSTATUS 1 Nov 05 08:19:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6516(8164451) done Nov 05 08:19:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166268 InitPrio=26952 usec=6203 Nov 05 08:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7167(8165785) WEXITSTATUS 1 Nov 05 08:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7167(8165785) done Nov 05 08:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154668 WEXITSTATUS 0 Nov 05 08:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154668 done Nov 05 08:20:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2197(8164894) WEXITSTATUS 0 Nov 05 08:20:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2197(8164894) done Nov 05 08:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2031(8164411) WEXITSTATUS 0 Nov 05 08:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2031(8164411) done Nov 05 08:20:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166262 in gpu on r808u08n01 Nov 05 08:20:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3888(8166269) in gpu on r4513u26n01 Nov 05 08:20:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154734 in day on r914u05n02 Nov 05 08:20:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2624(8166270) in day on r913u28n04 Nov 05 08:20:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2625(8166271) in day on r816u09n08 Nov 05 08:20:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2114(8164569) WEXITSTATUS 0 Nov 05 08:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2114(8164569) done Nov 05 08:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159110 WEXITSTATUS 0 Nov 05 08:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159110 done Nov 05 08:20:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166272 InitPrio=34694 usec=8348 Nov 05 08:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2026(8164395) WEXITSTATUS 0 Nov 05 08:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2026(8164395) done Nov 05 08:20:32 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166272 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7444(8166273) in scavenge on r801u14n01 Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7445(8166274) in scavenge on r904u01n03 Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7446(8166275) in scavenge on r917u13n04 Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7447(8166276) in scavenge on r802u06n03 Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7448(8166277) in scavenge on r802u09n04 Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7449(8166278) in scavenge on r906u39n01 Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7450(8166279) in scavenge on r907u28n01 Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7451(8166280) in scavenge on r805u02n01 Nov 05 08:20:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7452(8166281) in scavenge on r919u09n01 Nov 05 08:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2624(8152891) WEXITSTATUS 0 Nov 05 08:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2624(8152891) done Nov 05 08:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2175(8164799) WEXITSTATUS 0 Nov 05 08:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2175(8164799) done Nov 05 08:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2626(8166282) NodeList=r815u35n01 #CPUs=1 Partition=day Nov 05 08:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2627(8166283) NodeList=r816u09n06 #CPUs=1 Partition=day Nov 05 08:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166268 NodeList=r912u01n02 #CPUs=24 Partition=scavenge Nov 05 08:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7180(8165803) WEXITSTATUS 1 Nov 05 08:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7180(8165803) done Nov 05 08:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2190(8164867) WEXITSTATUS 0 Nov 05 08:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2190(8164867) done Nov 05 08:21:14 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2735(8128183) Nov 05 08:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7301(8165981) WEXITSTATUS 1 Nov 05 08:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7301(8165981) done Nov 05 08:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6793(8164949) WEXITSTATUS 1 Nov 05 08:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6793(8164949) done Nov 05 08:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7418(8166197) WEXITSTATUS 1 Nov 05 08:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7418(8166197) done Nov 05 08:21:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7441(8166256) WEXITSTATUS 1 Nov 05 08:21:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7441(8166256) done Nov 05 08:21:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:21:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2628(8166284) NodeList=r815u09n07 #CPUs=1 Partition=day Nov 05 08:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2208(8164910) WEXITSTATUS 0 Nov 05 08:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2208(8164910) done Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:21:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:22:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2001(8164218) WEXITSTATUS 0 Nov 05 08:22:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2001(8164218) done Nov 05 08:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2249(8165053) WEXITSTATUS 0 Nov 05 08:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2249(8165053) done Nov 05 08:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2228(8164991) WEXITSTATUS 0 Nov 05 08:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2228(8164991) done Nov 05 08:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2629(8166285) NodeList=r914u05n03 #CPUs=1 Partition=day Nov 05 08:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2630(8166286) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 08:22:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2631(8166287) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 08:22:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166288 NodeList=(null) usec=26599 Nov 05 08:23:00 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166288 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 08:23:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7453(8166289) in scavenge on r802u04n01 Nov 05 08:23:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7454(8166290) in scavenge on r911u24n03 Nov 05 08:23:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7455(8166291) in scavenge on r815u23n09 Nov 05 08:23:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7456(8166292) in scavenge on r918u05n03 Nov 05 08:23:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7457(8166293) in scavenge on r909u07n04 Nov 05 08:23:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7458(8166294) in scavenge on r909u10n04 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2632(8166295) in scavenge on r917u10n04 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2633(8166296) in scavenge on r917u10n04 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2634(8166297) in scavenge on r815u03n11 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2635(8166298) in scavenge on r815u03n12 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2636(8166299) in scavenge on r912u13n01 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2637(8166300) in scavenge on r806u31n01 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2638(8166301) in scavenge on r805u36n01 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2194(8164876) WEXITSTATUS 0 Nov 05 08:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2194(8164876) done Nov 05 08:23:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6255(8163496) WEXITSTATUS 1 Nov 05 08:23:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6255(8163496) done Nov 05 08:23:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820393 uid 18896 Nov 05 08:23:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_18(8158320) WEXITSTATUS 1 Nov 05 08:23:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_18(8158320) done Nov 05 08:23:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820392 uid 18896 Nov 05 08:23:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820391 uid 18896 Nov 05 08:23:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820390 uid 18896 Nov 05 08:23:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820389 uid 18896 Nov 05 08:23:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820388 uid 18896 Nov 05 08:23:30 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820388 Nov 05 08:23:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820388 sig=9 returned: Invalid job id specified Nov 05 08:23:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820387 uid 18896 Nov 05 08:23:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820386 uid 18896 Nov 05 08:23:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820385 uid 18896 Nov 05 08:23:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820384 uid 18896 Nov 05 08:23:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820383 uid 18896 Nov 05 08:23:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820383 uid 18896 Nov 05 08:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821338 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820382 uid 18896 Nov 05 08:23:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820381 uid 18896 Nov 05 08:23:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821339 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820380 uid 18896 Nov 05 08:23:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820379 uid 18896 Nov 05 08:23:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821340 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821341 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821342 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820378 uid 18896 Nov 05 08:23:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820376 uid 18896 Nov 05 08:23:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820377 uid 18896 Nov 05 08:23:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820375 uid 18896 Nov 05 08:23:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820374 uid 18896 Nov 05 08:23:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_547(7571512) Nov 05 08:23:44 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2741(8128193) Nov 05 08:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821343 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821344 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821345 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820373 uid 18896 Nov 05 08:23:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820372 uid 18896 Nov 05 08:23:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820371 uid 18896 Nov 05 08:23:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821346 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821347 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821348 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2639(8166302) NodeList=r919u01n03 #CPUs=1 Partition=day Nov 05 08:23:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2640(8166303) NodeList=r815u23n01 #CPUs=1 Partition=day Nov 05 08:23:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820370 uid 18896 Nov 05 08:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2137(8164652) WEXITSTATUS 0 Nov 05 08:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2137(8164652) done Nov 05 08:23:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 08:23:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820369 uid 18896 Nov 05 08:23:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821349 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821350 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821351 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821352 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820368 uid 18896 Nov 05 08:23:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820367 uid 18896 Nov 05 08:23:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820366 uid 18896 Nov 05 08:23:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820365 uid 18896 Nov 05 08:23:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820364 uid 18896 Nov 05 08:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821353 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821354 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820363 uid 18896 Nov 05 08:23:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820362 uid 18896 Nov 05 08:23:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820361 uid 18896 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:23:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820360 uid 18896 Nov 05 08:23:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821355 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821356 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821357 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821358 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821359 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820359 uid 18896 Nov 05 08:23:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821360 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821361 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821362 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 08:23:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820358 uid 18896 Nov 05 08:24:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820357 uid 18896 Nov 05 08:24:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159109 WEXITSTATUS 0 Nov 05 08:24:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159109 done Nov 05 08:24:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820356 uid 18896 Nov 05 08:24:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821363 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 08:24:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820355 uid 18896 Nov 05 08:24:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820354 uid 18896 Nov 05 08:24:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 08:24:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820353 uid 18896 Nov 05 08:24:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820352 uid 18896 Nov 05 08:24:04 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820352 Nov 05 08:24:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820352 sig=9 returned: Invalid job id specified Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821364 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821365 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821366 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821367 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821368 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820351 uid 18896 Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820351 Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820351 sig=9 returned: Invalid job id specified Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2043(8164457) WEXITSTATUS 0 Nov 05 08:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2043(8164457) done Nov 05 08:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2052(8164486) WEXITSTATUS 0 Nov 05 08:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2052(8164486) done Nov 05 08:24:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820350 uid 18896 Nov 05 08:24:06 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820350 Nov 05 08:24:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820350 sig=9 returned: Invalid job id specified Nov 05 08:24:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820349 uid 18896 Nov 05 08:24:08 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820349 Nov 05 08:24:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820349 sig=9 returned: Invalid job id specified Nov 05 08:24:09 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7821369 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:24:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820348 uid 18896 Nov 05 08:24:09 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820348 Nov 05 08:24:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820348 sig=9 returned: Invalid job id specified Nov 05 08:24:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820347 uid 18896 Nov 05 08:24:10 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820347 Nov 05 08:24:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820347 sig=9 returned: Invalid job id specified Nov 05 08:24:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=782034 uid 18896 Nov 05 08:24:11 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=782034 Nov 05 08:24:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=782034 sig=9 returned: Invalid job id specified Nov 05 08:24:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=782034 uid 18896 Nov 05 08:24:12 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=782034 Nov 05 08:24:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=782034 sig=9 returned: Invalid job id specified Nov 05 08:24:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820346 uid 18896 Nov 05 08:24:14 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820346 Nov 05 08:24:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820346 sig=9 returned: Invalid job id specified Nov 05 08:24:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820345 uid 18896 Nov 05 08:24:14 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820345 Nov 05 08:24:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820345 sig=9 returned: Invalid job id specified Nov 05 08:24:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820344 uid 18896 Nov 05 08:24:15 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820344 Nov 05 08:24:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820344 sig=9 returned: Invalid job id specified Nov 05 08:24:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820343 uid 18896 Nov 05 08:24:16 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820343 Nov 05 08:24:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820343 sig=9 returned: Invalid job id specified Nov 05 08:24:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820342 uid 18896 Nov 05 08:24:17 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820342 Nov 05 08:24:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820342 sig=9 returned: Invalid job id specified Nov 05 08:24:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820341 uid 18896 Nov 05 08:24:18 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820341 Nov 05 08:24:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820341 sig=9 returned: Invalid job id specified Nov 05 08:24:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820340 uid 18896 Nov 05 08:24:19 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820340 Nov 05 08:24:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820340 sig=9 returned: Invalid job id specified Nov 05 08:24:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820339 uid 18896 Nov 05 08:24:24 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820339 Nov 05 08:24:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820339 sig=9 returned: Invalid job id specified Nov 05 08:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1892(8162916) WEXITSTATUS 0 Nov 05 08:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1892(8162916) done Nov 05 08:24:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7820338 uid 18896 Nov 05 08:24:26 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7820338 Nov 05 08:24:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=7820338 sig=9 returned: Invalid job id specified Nov 05 08:24:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166304 InitPrio=58938 usec=8244 Nov 05 08:24:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166304 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 08:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2186(8164853) WEXITSTATUS 0 Nov 05 08:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2186(8164853) done Nov 05 08:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7235(8165888) WEXITSTATUS 1 Nov 05 08:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7235(8165888) done Nov 05 08:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2182(8164835) WEXITSTATUS 0 Nov 05 08:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2182(8164835) done Nov 05 08:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6989(8165354) WEXITSTATUS 1 Nov 05 08:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6989(8165354) done Nov 05 08:24:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2641(8166305) NodeList=r909u29n02 #CPUs=1 Partition=day Nov 05 08:24:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2642(8166306) NodeList=r815u35n04 #CPUs=1 Partition=day Nov 05 08:24:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2643(8166307) NodeList=r815u35n04 #CPUs=1 Partition=day Nov 05 08:24:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2644(8166308) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 08:24:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2645(8166309) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 08:24:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2646(8166310) NodeList=r816u15n05 #CPUs=1 Partition=day Nov 05 08:24:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7102(8165658) WEXITSTATUS 1 Nov 05 08:24:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7102(8165658) done Nov 05 08:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6863(8165076) WEXITSTATUS 1 Nov 05 08:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6863(8165076) done Nov 05 08:25:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2024(8164393) WEXITSTATUS 0 Nov 05 08:25:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2024(8164393) done Nov 05 08:25:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166311 NodeList=(null) usec=22919 Nov 05 08:25:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_146(7571110) WEXITSTATUS 0 Nov 05 08:25:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_146(7571110) done Nov 05 08:25:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_307(7572475) WEXITSTATUS 0 Nov 05 08:25:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_307(7572475) done Nov 05 08:25:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2057(8164512) WEXITSTATUS 0 Nov 05 08:25:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2057(8164512) done Nov 05 08:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5706(8162419) WEXITSTATUS 1 Nov 05 08:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5706(8162419) done Nov 05 08:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7428(8166217) WEXITSTATUS 1 Nov 05 08:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7428(8166217) done Nov 05 08:25:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2647(8166312) NodeList=r912u36n01 #CPUs=1 Partition=day Nov 05 08:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7459(8166313) in scavenge on r815u09n10 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7460(8166314) in scavenge on r903u34n01 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7461(8166315) in scavenge on r806u02n03 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7462(8166316) in scavenge on r806u06n01 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7463(8166317) in scavenge on r909u13n01 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7464(8166318) in scavenge on r909u13n04 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7465(8166319) in scavenge on r918u05n03 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7466(8166320) in scavenge on r919u01n03 Nov 05 08:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_286(7571251) in scavenge on r802u09n04 Nov 05 08:25:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2648(8166321) in scavenge on r912u05n04 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:26:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154704 WEXITSTATUS 0 Nov 05 08:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154704 done Nov 05 08:26:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154737 NodeList=r904u07n01 #CPUs=16 Partition=day Nov 05 08:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1956(8163539) WEXITSTATUS 0 Nov 05 08:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1956(8163539) done Nov 05 08:26:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166322 InitPrio=20041 usec=4305 Nov 05 08:26:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166118 WEXITSTATUS 0 Nov 05 08:26:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166118 done Nov 05 08:26:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166322 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01 #CPUs=288 Partition=day Nov 05 08:26:14 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2562(8127743) has been requeued to reclaim resources for JobId=Invalid Nov 05 08:26:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2562(8127743) Nov 05 08:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1893(8162957) WEXITSTATUS 0 Nov 05 08:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1893(8162957) done Nov 05 08:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7214(8165860) WEXITSTATUS 1 Nov 05 08:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7214(8165860) done Nov 05 08:26:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821377 uid 18896 Nov 05 08:26:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=782137 uid 18896 Nov 05 08:26:36 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=782137 Nov 05 08:26:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=782137 sig=9 returned: Invalid job id specified Nov 05 08:26:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7420(8166209) WEXITSTATUS 1 Nov 05 08:26:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7420(8166209) done Nov 05 08:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1895(8162959) WEXITSTATUS 0 Nov 05 08:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1895(8162959) done Nov 05 08:26:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2649(8166323) NodeList=r816u15n04 #CPUs=1 Partition=day Nov 05 08:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2219(8164967) WEXITSTATUS 0 Nov 05 08:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2219(8164967) done Nov 05 08:26:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2650(8166324) NodeList=r815u09n05 #CPUs=1 Partition=day Nov 05 08:26:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821376 uid 18896 Nov 05 08:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2223(8164971) WEXITSTATUS 0 Nov 05 08:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2223(8164971) done Nov 05 08:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2218(8164966) WEXITSTATUS 0 Nov 05 08:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2218(8164966) done Nov 05 08:26:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821375 uid 18896 Nov 05 08:26:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2651(8166325) NodeList=r816u15n08 #CPUs=1 Partition=day Nov 05 08:26:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2652(8166326) NodeList=r815u15n01 #CPUs=1 Partition=day Nov 05 08:26:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821374 uid 18896 Nov 05 08:26:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821373 uid 18896 Nov 05 08:26:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821372 uid 18896 Nov 05 08:26:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2171(8164782) WEXITSTATUS 0 Nov 05 08:26:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2171(8164782) done Nov 05 08:26:45 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166327 NodeList=(null) usec=65918 Nov 05 08:26:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821371 uid 18896 Nov 05 08:26:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2653(8166328) NodeList=r815u15n01 #CPUs=1 Partition=day Nov 05 08:26:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2654(8166329) NodeList=r815u15n05 #CPUs=1 Partition=day Nov 05 08:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2124(8164599) WEXITSTATUS 0 Nov 05 08:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2124(8164599) done Nov 05 08:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154652 WEXITSTATUS 0 Nov 05 08:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154652 done Nov 05 08:26:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154738 NodeList=r912u32n04 #CPUs=16 Partition=day Nov 05 08:26:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2655(8166330) NodeList=r909u04n04 #CPUs=1 Partition=day Nov 05 08:26:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821370 uid 18896 Nov 05 08:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2076(8164531) WEXITSTATUS 0 Nov 05 08:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2076(8164531) done Nov 05 08:27:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821369 uid 18896 Nov 05 08:27:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821364 uid 18896 Nov 05 08:27:11 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2579(8127772) has been requeued to reclaim resources for JobId=8166311 Nov 05 08:27:11 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2848(8128484) has been requeued to reclaim resources for JobId=8166311 Nov 05 08:27:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083896 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821365 uid 18896 Nov 05 08:27:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821366 uid 18896 Nov 05 08:27:12 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2579(8127772) Nov 05 08:27:13 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2848(8128484) Nov 05 08:27:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821367 uid 18896 Nov 05 08:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166311 NodeList=r815u09n12 #CPUs=20 Partition=pi_manohar Nov 05 08:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166327 NodeList=r815u09n10 #CPUs=10 Partition=pi_manohar Nov 05 08:27:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083897 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821368 uid 18896 Nov 05 08:27:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:27:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083898 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083899 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083900 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821363 uid 18896 Nov 05 08:27:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821362 uid 18896 Nov 05 08:27:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821361 uid 18896 Nov 05 08:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2143(8164669) WEXITSTATUS 0 Nov 05 08:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2143(8164669) done Nov 05 08:27:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083901 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083902 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2656(8166331) NodeList=r815u29n02 #CPUs=1 Partition=day Nov 05 08:27:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821360 uid 18896 Nov 05 08:27:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083903 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083904 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083905 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821359 uid 18896 Nov 05 08:27:31 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083906 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821358 uid 18896 Nov 05 08:27:33 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_receive_msgs: [[login1.grace.ycrc.yale.edu]:36549] failed: Socket timed out on send/recv operation Nov 05 08:27:33 r912u23n01 slurmctld[163760]: error: slurm_receive_msgs: [[login1.grace.ycrc.yale.edu]:36549] failed: Socket timed out on send/recv operation Nov 05 08:27:33 r912u23n01 slurmctld[163760]: slurmctld: Killing interactive JobId=8166311: Communication connection failure Nov 05 08:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166311 WEXITSTATUS 1 Nov 05 08:27:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166311 done Nov 05 08:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083907 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821357 uid 18896 Nov 05 08:27:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821356 uid 18896 Nov 05 08:27:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821355 uid 18896 Nov 05 08:27:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083908 NodeList=r807u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821354 uid 18896 Nov 05 08:27:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821353 uid 18896 Nov 05 08:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7256(8165918) WEXITSTATUS 1 Nov 05 08:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7256(8165918) done Nov 05 08:27:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821352 uid 18896 Nov 05 08:27:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821351 uid 18896 Nov 05 08:27:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083909 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083910 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821350 uid 18896 Nov 05 08:27:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2191(8164868) WEXITSTATUS 0 Nov 05 08:27:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2191(8164868) done Nov 05 08:27:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083911 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083912 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2657(8166332) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 08:27:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821349 uid 18896 Nov 05 08:27:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=78213488 uid 18896 Nov 05 08:27:45 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=78213488 Nov 05 08:27:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=18896 JobId=78213488 sig=9 returned: Invalid job id specified Nov 05 08:27:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083913 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083914 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083915 NodeList=r903u12n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083916 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821348 uid 18896 Nov 05 08:27:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821347 uid 18896 Nov 05 08:27:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821346 uid 18896 Nov 05 08:27:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821345 uid 18896 Nov 05 08:27:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083917 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083918 NodeList=r913u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821344 uid 18896 Nov 05 08:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7290(8165966) WEXITSTATUS 1 Nov 05 08:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7290(8165966) done Nov 05 08:27:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083919 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083920 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083921 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6136(8163177) WEXITSTATUS 1 Nov 05 08:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6136(8163177) done Nov 05 08:27:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821343 uid 18896 Nov 05 08:27:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821342 uid 18896 Nov 05 08:27:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821341 uid 18896 Nov 05 08:27:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083922 NodeList=r807u22n01 #CPUs=1 Partition=pi_ohern Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2075(8164530) WEXITSTATUS 0 Nov 05 08:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2075(8164530) done Nov 05 08:28:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821340 uid 18896 Nov 05 08:28:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083923 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 08:28:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083924 NodeList=r913u35n01 #CPUs=1 Partition=pi_ohern Nov 05 08:28:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821339 uid 18896 Nov 05 08:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154680 WEXITSTATUS 0 Nov 05 08:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154680 done Nov 05 08:28:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083925 NodeList=r903u24n01 #CPUs=1 Partition=pi_ohern Nov 05 08:28:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083926 NodeList=r903u26n01 #CPUs=1 Partition=pi_ohern Nov 05 08:28:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154739 NodeList=r913u24n02 #CPUs=16 Partition=day Nov 05 08:28:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7821338 uid 18896 Nov 05 08:28:09 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8083927 NodeList=r911u32n01 #CPUs=1 Partition=pi_ohern Nov 05 08:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2237(8165011) WEXITSTATUS 0 Nov 05 08:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2237(8165011) done Nov 05 08:28:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2658(8166333) NodeList=r815u23n06 #CPUs=1 Partition=day Nov 05 08:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154707 WEXITSTATUS 0 Nov 05 08:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154707 done Nov 05 08:28:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154740 NodeList=r919u09n01 #CPUs=16 Partition=day Nov 05 08:28:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2157(8164744) WEXITSTATUS 0 Nov 05 08:28:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2157(8164744) done Nov 05 08:28:32 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2659(8166334) NodeList=r815u03n03 #CPUs=1 Partition=day Nov 05 08:28:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7467(8166335) in scavenge on r801u06n01 Nov 05 08:28:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7468(8166336) in scavenge on r815u09n12 Nov 05 08:28:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7469(8166337) in scavenge on r815u09n12 Nov 05 08:28:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7470(8166338) in scavenge on r914u05n02 Nov 05 08:28:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7471(8166339) in scavenge on r909u07n04 Nov 05 08:28:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7472(8166340) in scavenge on r909u10n01 Nov 05 08:28:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7473(8166341) in scavenge on r913u09n03 Nov 05 08:28:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7474(8166342) in scavenge on r913u13n01 Nov 05 08:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7099(8165655) WEXITSTATUS 1 Nov 05 08:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7099(8165655) done Nov 05 08:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2086(8164541) WEXITSTATUS 0 Nov 05 08:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2086(8164541) done Nov 05 08:28:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166343 InitPrio=21719 usec=4813 Nov 05 08:28:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166343 NodeList=r807u37n01 #CPUs=5 Partition=bigmem Nov 05 08:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7352(8166061) WEXITSTATUS 1 Nov 05 08:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7352(8166061) done Nov 05 08:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3886(8166185) WEXITSTATUS 0 Nov 05 08:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3886(8166185) done Nov 05 08:28:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7471(8166339) WEXITSTATUS 1 Nov 05 08:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7471(8166339) done Nov 05 08:29:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6568(8164605) WEXITSTATUS 1 Nov 05 08:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6568(8164605) done Nov 05 08:29:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6003(8163008) WEXITSTATUS 1 Nov 05 08:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6003(8163008) done Nov 05 08:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7152(8165757) WEXITSTATUS 1 Nov 05 08:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7152(8165757) done Nov 05 08:29:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7452(8166281) WEXITSTATUS 1 Nov 05 08:29:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7452(8166281) done Nov 05 08:29:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7161(8165772) WEXITSTATUS 1 Nov 05 08:29:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7161(8165772) done Nov 05 08:29:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2174(8164798) WEXITSTATUS 0 Nov 05 08:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2174(8164798) done Nov 05 08:29:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2660(8166344) NodeList=r914u05n04 #CPUs=1 Partition=day Nov 05 08:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3888(8166269) WEXITSTATUS 0 Nov 05 08:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3888(8166269) done Nov 05 08:29:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7317(8166005) WEXITSTATUS 1 Nov 05 08:29:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7317(8166005) done Nov 05 08:29:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7422(8166211) WEXITSTATUS 1 Nov 05 08:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7422(8166211) done Nov 05 08:30:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:30:00 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 08:30:00 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:30:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_2(8158220) WEXITSTATUS 1 Nov 05 08:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_2(8158220) done Nov 05 08:30:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:30:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2661(8166345) NodeList=r909u04n01 #CPUs=1 Partition=day Nov 05 08:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 08:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 08:30:18 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 08:30:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166346 InitPrio=10612 usec=3847 Nov 05 08:30:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166347 InitPrio=10612 usec=3218 Nov 05 08:30:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166348 InitPrio=10612 usec=2849 Nov 05 08:30:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166349 InitPrio=10612 usec=2555 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166350 InitPrio=10612 usec=1799 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2077(8164532) WEXITSTATUS 0 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2077(8164532) done Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166351 InitPrio=10612 usec=2334 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166352 InitPrio=10612 usec=1984 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166353 InitPrio=10612 usec=1707 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166354 InitPrio=10612 usec=1486 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166355 InitPrio=10612 usec=1478 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166356 InitPrio=10612 usec=1488 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166357 InitPrio=10612 usec=1467 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166358 InitPrio=10612 usec=1516 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166359 InitPrio=10612 usec=1469 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166360 InitPrio=10612 usec=1461 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166361 InitPrio=10612 usec=1555 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166362 InitPrio=10612 usec=1527 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166363 InitPrio=10612 usec=1482 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166364 InitPrio=10612 usec=1481 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166365 InitPrio=10612 usec=1491 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166366 InitPrio=10612 usec=1458 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166367 InitPrio=10612 usec=1490 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166368 InitPrio=10612 usec=1481 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166369 InitPrio=10612 usec=1449 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166370 InitPrio=10612 usec=1473 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166371 InitPrio=10612 usec=1461 Nov 05 08:30:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166372 InitPrio=10612 usec=1453 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166373 InitPrio=10612 usec=1451 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166374 InitPrio=10612 usec=3485 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166375 InitPrio=10612 usec=2816 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166376 InitPrio=10612 usec=1750 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166377 InitPrio=10612 usec=1463 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166378 InitPrio=10612 usec=1498 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166379 InitPrio=10612 usec=1433 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166380 InitPrio=10612 usec=1474 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166381 InitPrio=10612 usec=1497 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166382 InitPrio=10612 usec=1538 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166383 InitPrio=10612 usec=1454 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166384 InitPrio=10612 usec=1449 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166385 InitPrio=10612 usec=1559 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166386 InitPrio=10612 usec=1528 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166387 InitPrio=10612 usec=1533 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166388 InitPrio=10612 usec=1468 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166389 InitPrio=10612 usec=1580 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166390 InitPrio=10612 usec=1479 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166391 InitPrio=10612 usec=1502 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166392 InitPrio=10612 usec=1468 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166393 InitPrio=10612 usec=1473 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166394 InitPrio=10612 usec=1475 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166395 InitPrio=10612 usec=1457 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166396 InitPrio=10612 usec=1438 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166397 InitPrio=10612 usec=1505 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166398 InitPrio=10612 usec=1499 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166399 InitPrio=10612 usec=1466 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166400 InitPrio=10612 usec=1565 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166401 InitPrio=10612 usec=1495 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166402 InitPrio=10612 usec=1498 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166403 InitPrio=10612 usec=1510 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166404 InitPrio=10612 usec=1462 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166405 InitPrio=10612 usec=1507 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166406 InitPrio=10612 usec=1473 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166407 InitPrio=10612 usec=1490 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166408 InitPrio=10612 usec=1531 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166409 InitPrio=10612 usec=1490 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166410 InitPrio=10612 usec=1465 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166411 InitPrio=10612 usec=1863 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166412 InitPrio=10612 usec=1910 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166413 InitPrio=10612 usec=4391 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166414 InitPrio=10612 usec=3144 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166415 InitPrio=10612 usec=2446 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166416 InitPrio=10612 usec=1844 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166417 InitPrio=10612 usec=1759 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166418 InitPrio=10612 usec=1511 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166419 InitPrio=10612 usec=1469 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166420 InitPrio=10612 usec=1474 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166421 InitPrio=10612 usec=1486 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166422 InitPrio=10612 usec=1463 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166423 InitPrio=10612 usec=1454 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166424 InitPrio=10612 usec=1596 Nov 05 08:30:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166425 InitPrio=10612 usec=1488 Nov 05 08:30:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:30:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2023(8164378) WEXITSTATUS 0 Nov 05 08:30:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2023(8164378) done Nov 05 08:30:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1936(8163362) WEXITSTATUS 0 Nov 05 08:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1936(8163362) done Nov 05 08:30:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3887(8166251) WEXITSTATUS 0 Nov 05 08:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3887(8166251) done Nov 05 08:30:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2002(8164219) WEXITSTATUS 0 Nov 05 08:30:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2002(8164219) done Nov 05 08:30:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:30:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2662(8166426) NodeList=r909u07n03 #CPUs=1 Partition=day Nov 05 08:30:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2663(8166427) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 08:30:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2664(8166428) NodeList=r816u15n07 #CPUs=1 Partition=day Nov 05 08:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159108 WEXITSTATUS 0 Nov 05 08:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159108 done Nov 05 08:30:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:31:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3889(8166429) in gpu on r808u08n01 Nov 05 08:31:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3890(8166430) in gpu on r808u14n01 Nov 05 08:31:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3891(8166431) in gpu on r4513u26n01 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166346 in gpu on r806u41n01 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166347 in gpu on r806u41n01 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166348 in gpu on r915u09n01 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166349 in gpu on r915u09n01 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166350 in gpu on r915u13n01 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166351 in gpu on r915u13n01 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166352 in gpu on r4513u32n01 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1037(7570699) WEXITSTATUS 0 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1037(7570699) done Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_601(7571566) WEXITSTATUS 0 Nov 05 08:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_601(7571566) done Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7475(8166432) in scavenge on r903u07n01 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7476(8166433) in scavenge on r815u09n11 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7477(8166434) in scavenge on r907u32n01 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7478(8166435) in scavenge on r807u11n01 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7479(8166436) in scavenge on r909u07n04 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7480(8166437) in scavenge on r914u05n03 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7481(8166438) in scavenge on r919u09n01 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7482(8166439) in scavenge on r805u09n02 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7483(8166440) in scavenge on r805u27n02 Nov 05 08:31:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7484(8166441) in scavenge on r913u24n02 Nov 05 08:31:15 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_685(7572854) Nov 05 08:31:15 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2750(8128222) Nov 05 08:31:15 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2752(8128224) Nov 05 08:31:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2665(8166442) in scavenge on r912u09n01 Nov 05 08:31:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2666(8166443) in scavenge on r816u23n04 Nov 05 08:31:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2667(8166444) in scavenge on r807u37n01 Nov 05 08:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7288(8165964) WEXITSTATUS 1 Nov 05 08:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7288(8165964) done Nov 05 08:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2188(8164855) WEXITSTATUS 0 Nov 05 08:31:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2188(8164855) done Nov 05 08:31:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2120(8164576) WEXITSTATUS 0 Nov 05 08:31:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2120(8164576) done Nov 05 08:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2108(8164563) WEXITSTATUS 0 Nov 05 08:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2108(8164563) done Nov 05 08:31:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2144(8164686) WEXITSTATUS 0 Nov 05 08:31:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2144(8164686) done Nov 05 08:31:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_21(8165257) WEXITSTATUS 0 Nov 05 08:31:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_21(8165257) done Nov 05 08:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7353(8166069) WEXITSTATUS 1 Nov 05 08:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7353(8166069) done Nov 05 08:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2668(8166445) NodeList=r919u01n04 #CPUs=1 Partition=day Nov 05 08:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2669(8166446) NodeList=r815u15n08 #CPUs=1 Partition=day Nov 05 08:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2670(8166447) NodeList=r816u03n04 #CPUs=1 Partition=day Nov 05 08:31:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2215(8164961) WEXITSTATUS 0 Nov 05 08:31:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2215(8164961) done Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:32:00 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3890(8166430) WEXITSTATUS 0 Nov 05 08:32:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3890(8166430) done Nov 05 08:32:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:16 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:32:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2119(8164575) WEXITSTATUS 0 Nov 05 08:32:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2119(8164575) done Nov 05 08:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2118(8164574) WEXITSTATUS 0 Nov 05 08:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2118(8164574) done Nov 05 08:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3889(8166429) WEXITSTATUS 0 Nov 05 08:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3889(8166429) done Nov 05 08:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 08:32:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 08:32:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 08:32:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2085(8164540) WEXITSTATUS 0 Nov 05 08:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2085(8164540) done Nov 05 08:32:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7259(8165925) WEXITSTATUS 1 Nov 05 08:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7259(8165925) done Nov 05 08:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2231(8164994) WEXITSTATUS 0 Nov 05 08:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2231(8164994) done Nov 05 08:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7479(8166436) WEXITSTATUS 1 Nov 05 08:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7479(8166436) done Nov 05 08:32:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7078(8165612) WEXITSTATUS 1 Nov 05 08:32:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7078(8165612) done Nov 05 08:32:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2093(8164548) WEXITSTATUS 0 Nov 05 08:32:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2093(8164548) done Nov 05 08:32:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8151653 WEXITSTATUS 130 Nov 05 08:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8151653 done Nov 05 08:32:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1437(8135970) WEXITSTATUS 0 Nov 05 08:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1437(8135970) done Nov 05 08:32:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2671(8166448) NodeList=r909u07n04 #CPUs=1 Partition=day Nov 05 08:32:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2672(8166449) NodeList=r918u09n02 #CPUs=1 Partition=day Nov 05 08:32:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2673(8166450) NodeList=r815u03n06 #CPUs=1 Partition=day Nov 05 08:32:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2674(8166451) NodeList=r815u15n04 #CPUs=1 Partition=day Nov 05 08:32:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8151651 WEXITSTATUS 130 Nov 05 08:32:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8151651 done Nov 05 08:32:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8151650 WEXITSTATUS 130 Nov 05 08:32:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8151650 done Nov 05 08:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7319(8166007) WEXITSTATUS 1 Nov 05 08:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7319(8166007) done Nov 05 08:32:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:32:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2011(8164299) WEXITSTATUS 0 Nov 05 08:32:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2011(8164299) done Nov 05 08:33:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2196(8164893) WEXITSTATUS 0 Nov 05 08:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2196(8164893) done Nov 05 08:33:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154692 WEXITSTATUS 0 Nov 05 08:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154692 done Nov 05 08:33:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7367(8166109) WEXITSTATUS 1 Nov 05 08:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7367(8166109) done Nov 05 08:33:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:33:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5973(8162973) WEXITSTATUS 1 Nov 05 08:33:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5973(8162973) done Nov 05 08:33:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2004(8164221) WEXITSTATUS 0 Nov 05 08:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2004(8164221) done Nov 05 08:33:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3884(8166130) WEXITSTATUS 0 Nov 05 08:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3884(8166130) done Nov 05 08:33:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_1173(8115442) Nov 05 08:33:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:33:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7015(8165420) WEXITSTATUS 1 Nov 05 08:33:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7015(8165420) done Nov 05 08:33:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:33:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154741 NodeList=r816u09n04 #CPUs=16 Partition=day Nov 05 08:33:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2675(8166452) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 08:33:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2676(8166453) NodeList=r919u05n04 #CPUs=1 Partition=day Nov 05 08:33:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2677(8166454) NodeList=r919u32n02 #CPUs=1 Partition=day Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3892(8166455) in gpu on r903u34n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3893(8166456) in gpu on r808u08n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3894(8166457) in gpu on r808u14n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166353 in gpu on r4513u23n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166354 in gpu on r4513u23n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166355 in gpu on r907u32n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166356 in gpu on r805u32n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166357 in gpu on r805u32n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166358 in gpu on r805u32n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166359 in gpu on r805u32n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166360 in gpu on r805u38n01 Nov 05 08:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166361 in gpu on r805u38n01 Nov 05 08:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154677 WEXITSTATUS 0 Nov 05 08:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154677 done Nov 05 08:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_385(7572553) WEXITSTATUS 0 Nov 05 08:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_385(7572553) done Nov 05 08:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_842(7573011) WEXITSTATUS 0 Nov 05 08:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_842(7573011) done Nov 05 08:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1017(7573186) WEXITSTATUS 0 Nov 05 08:33:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1017(7573186) done Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7485(8166458) in scavenge on r802u04n02 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7486(8166459) in scavenge on r802u04n04 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7487(8166460) in scavenge on r917u07n04 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7488(8166461) in scavenge on r917u13n02 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7489(8166462) in scavenge on r917u29n03 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7490(8166463) in scavenge on r909u07n04 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7491(8166464) in scavenge on r912u28n02 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7492(8166465) in scavenge on r912u36n02 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7493(8166466) in scavenge on r912u36n02 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7494(8166467) in scavenge on r913u28n02 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7495(8166468) in scavenge on r816u23n05 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_999(7571965) in scavenge on r802u09n02 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_655(7572824) in scavenge on r907u28n01 Nov 05 08:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_92(7572259) in scavenge on r907u32n01 Nov 05 08:33:58 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:900 Nov 05 08:33:59 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 08:34:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2678(8166469) in scavenge on r912u09n01 Nov 05 08:34:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2679(8166470) in scavenge on r906u13n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:34:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3894(8166457) WEXITSTATUS 0 Nov 05 08:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3894(8166457) done Nov 05 08:34:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7010(8165415) WEXITSTATUS 1 Nov 05 08:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7010(8165415) done Nov 05 08:34:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:34:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7486(8166459) WEXITSTATUS 1 Nov 05 08:34:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7486(8166459) done Nov 05 08:34:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:34:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651307_2(7651772) Nov 05 08:34:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7184(8165809) WEXITSTATUS 1 Nov 05 08:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7184(8165809) done Nov 05 08:34:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:34:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1929(8163308) WEXITSTATUS 0 Nov 05 08:34:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1929(8163308) done Nov 05 08:34:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:34:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7274(8165944) WEXITSTATUS 1 Nov 05 08:34:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7274(8165944) done Nov 05 08:34:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:34:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:34:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154742 NodeList=r913u05n03 #CPUs=16 Partition=day Nov 05 08:34:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2680(8166471) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 08:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160536 WEXITSTATUS 0 Nov 05 08:35:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160536 done Nov 05 08:35:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:35:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2041(8164448) WEXITSTATUS 0 Nov 05 08:35:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2041(8164448) done Nov 05 08:35:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:35:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166472 InitPrio=59533 usec=7410 Nov 05 08:35:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166472 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 08:35:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2010(8164298) WEXITSTATUS 0 Nov 05 08:35:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2010(8164298) done Nov 05 08:35:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:35:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2681(8166473) NodeList=r919u05n01 #CPUs=1 Partition=day Nov 05 08:35:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2682(8166474) NodeList=r815u15n08 #CPUs=1 Partition=day Nov 05 08:35:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166475 InitPrio=51292 usec=4446 Nov 05 08:35:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:35:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157950 WEXITSTATUS 0 Nov 05 08:35:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8157950 done Nov 05 08:35:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_145(7572313) WEXITSTATUS 0 Nov 05 08:35:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_145(7572313) done Nov 05 08:35:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_134(7572301) WEXITSTATUS 0 Nov 05 08:35:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_134(7572301) done Nov 05 08:35:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:35:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2109(8164564) WEXITSTATUS 0 Nov 05 08:35:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2109(8164564) done Nov 05 08:36:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7281(8165954) WEXITSTATUS 1 Nov 05 08:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7281(8165954) done Nov 05 08:36:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:36:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166476 InitPrio=34694 usec=7535 Nov 05 08:36:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:36:09 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 26141: QOSMaxSubmitJobPerUserLimit Nov 05 08:36:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 08:36:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7470(8166338) WEXITSTATUS 1 Nov 05 08:36:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7470(8166338) done Nov 05 08:36:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3895(8166477) in gpu on r808u14n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166362 in gpu on r4513u26n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166363 in gpu on r4513u26n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166364 in gpu on r4513u32n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166365 in gpu on r805u34n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166366 in gpu on r805u34n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166367 in gpu on r805u34n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166368 in gpu on r805u34n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166369 in gpu on r805u36n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7651307_2(7651772) in gpu on r907u34n01 Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 26141: QOSMaxSubmitJobPerUserLimit Nov 05 08:36:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 08:36:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 08:36:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7496(8166478) in scavenge on r802u04n04 Nov 05 08:36:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7497(8166479) in scavenge on r816u09n10 Nov 05 08:36:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7498(8166480) in scavenge on r913u01n02 Nov 05 08:36:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7499(8166481) in scavenge on r919u09n01 Nov 05 08:36:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7500(8166482) in scavenge on r805u06n03 Nov 05 08:36:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7501(8166483) in scavenge on r805u06n03 Nov 05 08:36:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_246(7572414) in scavenge on r805u27n02 Nov 05 08:36:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2683(8166484) in scavenge on r4513u23n01 Nov 05 08:36:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2684(8166485) in scavenge on r903u34n01 Nov 05 08:36:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2685(8166486) in scavenge on r906u13n01 Nov 05 08:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3892(8166455) WEXITSTATUS 0 Nov 05 08:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3892(8166455) done Nov 05 08:36:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2055(8164510) WEXITSTATUS 0 Nov 05 08:36:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2055(8164510) done Nov 05 08:36:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:36:45 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3566(8136078) has been requeued to reclaim resources for JobId=Invalid Nov 05 08:36:45 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4325(8154389) has been requeued to reclaim resources for JobId=Invalid Nov 05 08:36:46 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3566(8136078) Nov 05 08:36:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4325(8154389) Nov 05 08:36:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166475 NodeList=r911u01n02 #CPUs=36 Partition=pi_poland Nov 05 08:36:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:36:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:37:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6917(8165177) WEXITSTATUS 1 Nov 05 08:37:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6917(8165177) done Nov 05 08:37:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154700 WEXITSTATUS 0 Nov 05 08:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154700 done Nov 05 08:37:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:37:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7454(8166290) WEXITSTATUS 1 Nov 05 08:37:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7454(8166290) done Nov 05 08:37:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:37:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:37:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:37:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6252(8163493) WEXITSTATUS 1 Nov 05 08:37:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6252(8163493) done Nov 05 08:37:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:37:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166288 uid 26141 Nov 05 08:37:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166487 NodeList=(null) usec=25641 Nov 05 08:37:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166487 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 08:37:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:37:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2233(8165006) WEXITSTATUS 0 Nov 05 08:37:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2233(8165006) done Nov 05 08:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2039(8164440) WEXITSTATUS 0 Nov 05 08:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2039(8164440) done Nov 05 08:38:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2003(8164220) WEXITSTATUS 0 Nov 05 08:38:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2003(8164220) done Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:38:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:38:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7322(8166013) WEXITSTATUS 1 Nov 05 08:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7322(8166013) done Nov 05 08:38:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2013(8164325) WEXITSTATUS 0 Nov 05 08:38:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2013(8164325) done Nov 05 08:38:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7356(8166072) WEXITSTATUS 1 Nov 05 08:38:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7356(8166072) done Nov 05 08:38:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2056(8164511) WEXITSTATUS 0 Nov 05 08:38:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2056(8164511) done Nov 05 08:38:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7293(8165973) WEXITSTATUS 1 Nov 05 08:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7293(8165973) done Nov 05 08:38:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7474(8166342) WEXITSTATUS 1 Nov 05 08:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7474(8166342) done Nov 05 08:38:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3869(8165987) WEXITSTATUS 0 Nov 05 08:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3869(8165987) done Nov 05 08:38:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2169(8164774) WEXITSTATUS 0 Nov 05 08:39:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2169(8164774) done Nov 05 08:39:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3896(8166488) in gpu on r4513u29n01 Nov 05 08:39:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3897(8166489) in gpu on r903u34n01 Nov 05 08:39:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154745 in day on r815u29n02 Nov 05 08:39:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2686(8166490) in day on r805u25n03 Nov 05 08:39:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2687(8166491) in day on r918u09n03 Nov 05 08:39:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2688(8166492) in day on r919u05n03 Nov 05 08:39:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2689(8166493) in day on r815u35n03 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7502(8166494) in scavenge on r908u28n02 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7503(8166495) in scavenge on r911u01n01 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7504(8166496) in scavenge on r911u24n03 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7505(8166497) in scavenge on r815u03n11 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7506(8166498) in scavenge on r815u29n08 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7507(8166499) in scavenge on r816u23n07 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7508(8166500) in scavenge on r913u13n01 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7509(8166501) in scavenge on r914u01n02 Nov 05 08:39:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7510(8166502) in scavenge on r914u01n02 Nov 05 08:39:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3895(8166477) WEXITSTATUS 0 Nov 05 08:39:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3895(8166477) done Nov 05 08:39:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2146(8164705) WEXITSTATUS 0 Nov 05 08:39:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2146(8164705) done Nov 05 08:39:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2250(8165054) WEXITSTATUS 0 Nov 05 08:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2250(8165054) done Nov 05 08:39:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6329(8163793) WEXITSTATUS 1 Nov 05 08:39:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6329(8163793) done Nov 05 08:39:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3897(8166489) WEXITSTATUS 0 Nov 05 08:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3897(8166489) done Nov 05 08:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7250(8165912) WEXITSTATUS 1 Nov 05 08:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7250(8165912) done Nov 05 08:39:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2110(8164565) WEXITSTATUS 0 Nov 05 08:39:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2110(8164565) done Nov 05 08:39:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2200(8164902) WEXITSTATUS 0 Nov 05 08:39:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2200(8164902) done Nov 05 08:39:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2032(8164412) WEXITSTATUS 0 Nov 05 08:39:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2032(8164412) done Nov 05 08:39:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:39:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2115(8164570) WEXITSTATUS 0 Nov 05 08:39:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2115(8164570) done Nov 05 08:39:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2087(8164542) WEXITSTATUS 0 Nov 05 08:39:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2087(8164542) done Nov 05 08:39:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:40:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7509(8166501) WEXITSTATUS 1 Nov 05 08:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7509(8166501) done Nov 05 08:40:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:40:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7061(8165524) WEXITSTATUS 1 Nov 05 08:40:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7061(8165524) done Nov 05 08:40:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:40:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2217(8164965) WEXITSTATUS 0 Nov 05 08:40:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2217(8164965) done Nov 05 08:40:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:40:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2172(8164783) WEXITSTATUS 0 Nov 05 08:40:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2172(8164783) done Nov 05 08:40:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2230(8164993) WEXITSTATUS 0 Nov 05 08:40:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2230(8164993) done Nov 05 08:40:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:40:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1897(8163022) WEXITSTATUS 0 Nov 05 08:40:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1897(8163022) done Nov 05 08:40:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:40:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6644(8164721) WEXITSTATUS 1 Nov 05 08:40:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6644(8164721) done Nov 05 08:40:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:40:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2054(8164509) WEXITSTATUS 0 Nov 05 08:41:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2054(8164509) done Nov 05 08:41:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2222(8164970) WEXITSTATUS 0 Nov 05 08:41:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2222(8164970) done Nov 05 08:41:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2096(8164551) WEXITSTATUS 0 Nov 05 08:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2096(8164551) done Nov 05 08:41:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154685 WEXITSTATUS 0 Nov 05 08:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154685 done Nov 05 08:41:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2210(8164935) WEXITSTATUS 0 Nov 05 08:41:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2210(8164935) done Nov 05 08:41:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2037(8164427) WEXITSTATUS 0 Nov 05 08:41:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2037(8164427) done Nov 05 08:41:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2776(8128267) Nov 05 08:41:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2234(8165007) WEXITSTATUS 0 Nov 05 08:41:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2234(8165007) done Nov 05 08:41:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3898(8166503) in gpu on r808u14n01 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3899(8166504) in gpu on r903u34n01 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154743 in day on r913u13n02 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2690(8166505) in day on r805u06n01 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2691(8166506) in day on r909u10n01 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2692(8166507) in day on r913u13n03 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2693(8166508) in day on r918u01n01 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2694(8166509) in day on r918u05n02 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2695(8166510) in day on r815u35n03 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2696(8166511) in day on r815u09n08 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2697(8166512) in day on r815u15n01 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2698(8166513) in day on r816u15n04 Nov 05 08:41:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2699(8166514) in day on r816u15n08 Nov 05 08:41:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2239(8165021) WEXITSTATUS 0 Nov 05 08:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2239(8165021) done Nov 05 08:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2214(8164960) WEXITSTATUS 0 Nov 05 08:41:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2214(8164960) done Nov 05 08:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7511(8166515) in scavenge on r802u09n04 Nov 05 08:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7512(8166516) in scavenge on r903u07n01 Nov 05 08:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7513(8166517) in scavenge on r815u09n11 Nov 05 08:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7514(8166518) in scavenge on r805u06n04 Nov 05 08:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7515(8166519) in scavenge on r805u23n04 Nov 05 08:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7516(8166520) in scavenge on r914u01n02 Nov 05 08:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2059(8164514) WEXITSTATUS 0 Nov 05 08:41:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2059(8164514) done Nov 05 08:42:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2060(8164515) WEXITSTATUS 0 Nov 05 08:42:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2060(8164515) done Nov 05 08:42:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2432(8127447) WEXITSTATUS 1 Nov 05 08:42:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2432(8127447) done Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:42:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:42:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2220(8164968) WEXITSTATUS 0 Nov 05 08:42:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2220(8164968) done Nov 05 08:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2221(8164969) WEXITSTATUS 0 Nov 05 08:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2221(8164969) done Nov 05 08:42:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166476 NodeList=r816u15n08 #CPUs=1 Partition=day Nov 05 08:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1955(8163538) WEXITSTATUS 0 Nov 05 08:42:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1955(8163538) done Nov 05 08:42:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2018(8164361) WEXITSTATUS 0 Nov 05 08:42:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2018(8164361) done Nov 05 08:42:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2700(8166521) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 08:42:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2701(8166522) NodeList=r815u35n04 #CPUs=1 Partition=day Nov 05 08:42:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2702(8166523) NodeList=r815u03n05 #CPUs=1 Partition=day Nov 05 08:42:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2703(8166524) NodeList=r815u15n05 #CPUs=1 Partition=day Nov 05 08:42:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2091(8164546) WEXITSTATUS 0 Nov 05 08:42:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2091(8164546) done Nov 05 08:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2189(8164866) WEXITSTATUS 0 Nov 05 08:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2189(8164866) done Nov 05 08:43:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_10(8165230) WEXITSTATUS 0 Nov 05 08:43:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_10(8165230) done Nov 05 08:43:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:43:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2049(8164483) WEXITSTATUS 0 Nov 05 08:43:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2049(8164483) done Nov 05 08:43:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2704(8166525) NodeList=r805u11n02 #CPUs=1 Partition=day Nov 05 08:43:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2705(8166526) NodeList=r909u13n02 #CPUs=1 Partition=day Nov 05 08:43:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2111(8164566) WEXITSTATUS 0 Nov 05 08:43:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2111(8164566) done Nov 05 08:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7417(8166196) WEXITSTATUS 1 Nov 05 08:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7417(8166196) done Nov 05 08:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1954(8163537) WEXITSTATUS 0 Nov 05 08:43:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1954(8163537) done Nov 05 08:43:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154706 WEXITSTATUS 0 Nov 05 08:43:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154706 done Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:44:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:44:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_551(7572720) Nov 05 08:44:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_375(8114642) Nov 05 08:44:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154710 WEXITSTATUS 0 Nov 05 08:44:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154710 done Nov 05 08:44:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154744 in day on r913u28n03 Nov 05 08:44:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154748 in day on r816u03n05 Nov 05 08:44:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2706(8166527) in day on r815u15n05 Nov 05 08:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7153(8165758) WEXITSTATUS 1 Nov 05 08:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7153(8165758) done Nov 05 08:44:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7517(8166528) in scavenge on r816u23n07 Nov 05 08:44:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7518(8166529) in scavenge on r805u04n03 Nov 05 08:44:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7519(8166530) in scavenge on r914u05n02 Nov 05 08:44:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_292(7572460) in scavenge on r919u01n02 Nov 05 08:44:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_225(8114491) in scavenge on r816u03n01 Nov 05 08:44:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7275(8165948) WEXITSTATUS 1 Nov 05 08:44:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7275(8165948) done Nov 05 08:44:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7504(8166496) WEXITSTATUS 1 Nov 05 08:44:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7504(8166496) done Nov 05 08:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2153(8164727) WEXITSTATUS 0 Nov 05 08:44:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2153(8164727) done Nov 05 08:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2136(8164649) WEXITSTATUS 0 Nov 05 08:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2136(8164649) done Nov 05 08:45:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166487 uid 26141 Nov 05 08:45:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2216(8164964) WEXITSTATUS 0 Nov 05 08:45:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2216(8164964) done Nov 05 08:45:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166531 NodeList=(null) usec=17190 Nov 05 08:45:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166531 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 08:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7042(8165476) WEXITSTATUS 1 Nov 05 08:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7042(8165476) done Nov 05 08:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6960(8165280) WEXITSTATUS 1 Nov 05 08:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6960(8165280) done Nov 05 08:45:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2707(8166532) NodeList=r909u07n02 #CPUs=1 Partition=day Nov 05 08:45:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2708(8166533) NodeList=r815u15n01 #CPUs=1 Partition=day Nov 05 08:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2105(8164560) WEXITSTATUS 0 Nov 05 08:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2105(8164560) done Nov 05 08:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3878(8166066) WEXITSTATUS 0 Nov 05 08:45:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3878(8166066) done Nov 05 08:45:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:45:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2244(8165027) WEXITSTATUS 0 Nov 05 08:45:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2244(8165027) done Nov 05 08:45:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:46:01 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:46:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2053(8164508) WEXITSTATUS 0 Nov 05 08:46:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2053(8164508) done Nov 05 08:46:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7375(8166122) WEXITSTATUS 1 Nov 05 08:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7375(8166122) done Nov 05 08:46:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:46:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3885(8166184) WEXITSTATUS 0 Nov 05 08:46:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3885(8166184) done Nov 05 08:46:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:46:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2605(8152872) WEXITSTATUS 0 Nov 05 08:46:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2605(8152872) done Nov 05 08:46:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3896(8166488) WEXITSTATUS 0 Nov 05 08:46:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3896(8166488) done Nov 05 08:46:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3891(8166431) WEXITSTATUS 0 Nov 05 08:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3891(8166431) done Nov 05 08:46:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:46:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2709(8166534) NodeList=r815u03n08 #CPUs=1 Partition=day Nov 05 08:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3900(8166535) in gpu on r903u34n01 Nov 05 08:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3901(8166536) in gpu on r903u34n01 Nov 05 08:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3902(8166537) in gpu on r4513u26n01 Nov 05 08:47:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3903(8166538) in gpu on r4513u29n01 Nov 05 08:47:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:47:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 08:47:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7013(8165418) WEXITSTATUS 1 Nov 05 08:47:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7013(8165418) done Nov 05 08:47:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7520(8166539) in scavenge on r802u09n02 Nov 05 08:47:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7521(8166540) in scavenge on r911u24n03 Nov 05 08:47:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7522(8166541) in scavenge on r917u13n02 Nov 05 08:47:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7523(8166542) in scavenge on r816u23n11 Nov 05 08:47:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7524(8166543) in scavenge on r909u13n03 Nov 05 08:47:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7525(8166544) in scavenge on r914u05n04 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2710(8166545) in scavenge on r911u05n02 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2711(8166546) in scavenge on r801u23n03 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2712(8166547) in scavenge on r801u27n02 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2713(8166548) in scavenge on r815u29n11 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2714(8166549) in scavenge on r815u35n09 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2715(8166550) in scavenge on r815u35n09 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2716(8166551) in scavenge on r912u09n01 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2717(8166552) in scavenge on r912u13n01 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2718(8166553) in scavenge on r906u28n01 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2719(8166554) in scavenge on r806u31n01 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2720(8166555) in scavenge on r906u32n01 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2721(8166556) in scavenge on r906u32n01 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2722(8166557) in scavenge on r906u32n01 Nov 05 08:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2723(8166558) in scavenge on r805u38n01 Nov 05 08:47:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2693(8153080) WEXITSTATUS 0 Nov 05 08:47:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2693(8153080) done Nov 05 08:47:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2242(8165025) WEXITSTATUS 0 Nov 05 08:47:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2242(8165025) done Nov 05 08:47:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2025(8164394) WEXITSTATUS 0 Nov 05 08:47:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2025(8164394) done Nov 05 08:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3901(8166536) WEXITSTATUS 0 Nov 05 08:47:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3901(8166536) done Nov 05 08:47:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:47:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2724(8166559) NodeList=r919u01n02 #CPUs=1 Partition=day Nov 05 08:47:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2725(8166560) NodeList=r815u35n06 #CPUs=1 Partition=day Nov 05 08:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2162(8164767) WEXITSTATUS 0 Nov 05 08:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2162(8164767) done Nov 05 08:47:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159206 WEXITSTATUS 0 Nov 05 08:48:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159206 done Nov 05 08:48:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7330(8166027) WEXITSTATUS 1 Nov 05 08:48:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7330(8166027) done Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:48:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_64(8135763) WEXITSTATUS 0 Nov 05 08:48:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_64(8135763) done Nov 05 08:48:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:48:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7491(8166464) WEXITSTATUS 1 Nov 05 08:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7491(8166464) done Nov 05 08:48:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166561 InitPrio=28336 usec=7385 Nov 05 08:48:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2044(8164468) WEXITSTATUS 0 Nov 05 08:48:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2044(8164468) done Nov 05 08:48:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166561 uid 25715 Nov 05 08:48:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7341(8166045) WEXITSTATUS 1 Nov 05 08:48:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7341(8166045) done Nov 05 08:48:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:48:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2726(8166562) NodeList=r815u29n04 #CPUs=1 Partition=day Nov 05 08:48:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166563 InitPrio=28336 usec=5623 Nov 05 08:48:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:08 r912u23n01 slurmctld[163760]: slurmctld: error: accounts_list_build: No association for account wv48-ct-hmis Nov 05 08:49:08 r912u23n01 slurmctld[163760]: error: accounts_list_build: No association for account wv48-ct-hmis Nov 05 08:49:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7234(8165887) WEXITSTATUS 1 Nov 05 08:49:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7234(8165887) done Nov 05 08:49:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2789(8128313) Nov 05 08:49:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8164877 Nov 05 08:49:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2089(8164544) WEXITSTATUS 0 Nov 05 08:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2089(8164544) done Nov 05 08:49:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1899(8163039) WEXITSTATUS 0 Nov 05 08:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1899(8163039) done Nov 05 08:49:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:28 r912u23n01 slurmctld[163760]: slurmctld: error: accounts_list_build: No association for account wv48-ct-hmis Nov 05 08:49:28 r912u23n01 slurmctld[163760]: error: accounts_list_build: No association for account wv48-ct-hmis Nov 05 08:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1931(8163310) WEXITSTATUS 0 Nov 05 08:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1931(8163310) done Nov 05 08:49:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7455(8166291) WEXITSTATUS 1 Nov 05 08:49:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7455(8166291) done Nov 05 08:49:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2179(8164819) WEXITSTATUS 0 Nov 05 08:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2179(8164819) done Nov 05 08:49:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6959(8165279) WEXITSTATUS 1 Nov 05 08:49:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6959(8165279) done Nov 05 08:49:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6676680_6(6676680) Nov 05 08:49:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2727(8166564) NodeList=r805u23n01 #CPUs=1 Partition=day Nov 05 08:49:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2728(8166565) NodeList=r815u09n04 #CPUs=1 Partition=day Nov 05 08:49:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2729(8166566) NodeList=r816u03n05 #CPUs=1 Partition=day Nov 05 08:49:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166567 InitPrio=25489 usec=8029 Nov 05 08:49:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:49:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166563 in gpu on r903u34n01 Nov 05 08:49:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3904(8166568) in gpu on r808u11n01 Nov 05 08:49:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7310(8165994) WEXITSTATUS 1 Nov 05 08:49:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7310(8165994) done Nov 05 08:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7526(8166569) in scavenge on r916u04n01 Nov 05 08:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7527(8166570) in scavenge on r815u23n09 Nov 05 08:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7528(8166571) in scavenge on r816u29n11 Nov 05 08:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7529(8166572) in scavenge on r4513u20n01 Nov 05 08:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7530(8166573) in scavenge on r909u13n01 Nov 05 08:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7531(8166574) in scavenge on r912u28n02 Nov 05 08:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7532(8166575) in scavenge on r914u05n04 Nov 05 08:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7533(8166576) in scavenge on r918u01n03 Nov 05 08:50:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7247(8165904) WEXITSTATUS 1 Nov 05 08:50:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7247(8165904) done Nov 05 08:50:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2730(8166577) in scavenge on r912u13n01 Nov 05 08:50:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2731(8166578) in scavenge on r906u32n01 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166579 InitPrio=51282 usec=5026 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166580 InitPrio=10596 usec=3933 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166581 InitPrio=10596 usec=3903 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166582 InitPrio=10596 usec=5110 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166583 InitPrio=10596 usec=4695 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166584 InitPrio=10596 usec=4039 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166585 InitPrio=10596 usec=3921 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166586 InitPrio=10596 usec=2750 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166587 InitPrio=10596 usec=2566 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166588 InitPrio=10596 usec=3455 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166589 InitPrio=10596 usec=2521 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166590 InitPrio=10596 usec=3206 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166591 InitPrio=10596 usec=2474 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166592 InitPrio=10596 usec=2816 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166593 InitPrio=10596 usec=2899 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166594 InitPrio=10596 usec=2716 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166595 InitPrio=10596 usec=2917 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166596 InitPrio=10596 usec=2303 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166597 InitPrio=10596 usec=3549 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166598 InitPrio=10596 usec=2904 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166599 InitPrio=10596 usec=2251 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166600 InitPrio=10596 usec=2165 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166601 InitPrio=10596 usec=2134 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166602 InitPrio=10596 usec=2127 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166603 InitPrio=10596 usec=2110 Nov 05 08:50:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166604 InitPrio=10596 usec=2651 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:50:08 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:50:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1913(8163233) WEXITSTATUS 0 Nov 05 08:50:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1913(8163233) done Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166605 InitPrio=10596 usec=3846 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_369(8114636) WEXITSTATUS 0 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_369(8114636) done Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166606 InitPrio=10596 usec=3333 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166607 InitPrio=10596 usec=3047 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166608 InitPrio=10596 usec=2191 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166609 InitPrio=10596 usec=1939 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166610 InitPrio=10596 usec=1921 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166611 InitPrio=10596 usec=1926 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166612 InitPrio=10596 usec=1928 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166613 InitPrio=10596 usec=2003 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166614 InitPrio=10596 usec=1887 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166615 InitPrio=10596 usec=4332 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166616 InitPrio=10596 usec=3287 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166617 InitPrio=10596 usec=2736 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166618 InitPrio=10596 usec=2021 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166619 InitPrio=10596 usec=1887 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166620 InitPrio=10596 usec=1902 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166621 InitPrio=10596 usec=1891 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166622 InitPrio=10596 usec=1952 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166623 InitPrio=10596 usec=1932 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166624 InitPrio=10596 usec=1894 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166625 InitPrio=10596 usec=1942 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166626 InitPrio=10596 usec=1871 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166627 InitPrio=10596 usec=1935 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166628 InitPrio=10596 usec=1896 Nov 05 08:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166629 InitPrio=10596 usec=1890 Nov 05 08:50:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166567 NodeList=r815u23n04 #CPUs=24 Partition=day Nov 05 08:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2048(8164482) WEXITSTATUS 0 Nov 05 08:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2048(8164482) done Nov 05 08:50:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3904(8166568) WEXITSTATUS 0 Nov 05 08:50:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3904(8166568) done Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2008(8164296) WEXITSTATUS 0 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2008(8164296) done Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166630 InitPrio=10596 usec=3747 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166631 InitPrio=10596 usec=3277 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166632 InitPrio=10596 usec=2325 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166633 InitPrio=10596 usec=2019 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166634 InitPrio=10596 usec=1891 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166635 InitPrio=10596 usec=1909 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166636 InitPrio=10596 usec=1930 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166637 InitPrio=10596 usec=2781 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166638 InitPrio=10596 usec=2836 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166639 InitPrio=10596 usec=2320 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166640 InitPrio=10596 usec=2123 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166641 InitPrio=10596 usec=1925 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166642 InitPrio=10596 usec=1928 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166643 InitPrio=10596 usec=1944 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166644 InitPrio=10596 usec=1922 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166645 InitPrio=10596 usec=1866 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166646 InitPrio=10596 usec=2016 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166647 InitPrio=10596 usec=1942 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166648 InitPrio=10596 usec=1921 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166649 InitPrio=10596 usec=1906 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166650 InitPrio=10596 usec=1911 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166651 InitPrio=10596 usec=1895 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166652 InitPrio=10596 usec=1988 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166653 InitPrio=10596 usec=1864 Nov 05 08:50:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166654 InitPrio=10596 usec=1850 Nov 05 08:50:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:50:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2163(8164768) WEXITSTATUS 0 Nov 05 08:50:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2163(8164768) done Nov 05 08:50:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:50:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2824(8128414) has been requeued to reclaim resources for JobId=Invalid Nov 05 08:50:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3454(8135310) has been requeued to reclaim resources for JobId=Invalid Nov 05 08:50:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4602(8156178) has been requeued to reclaim resources for JobId=Invalid Nov 05 08:50:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8162530 Nov 05 08:50:47 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2268752700]] slurm_bufs_sendto(msg_type=RESPONSE_SLURM_RC) failed: Unexpected missing socket error Nov 05 08:50:47 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2268752701]] slurm_bufs_sendto(msg_type=RESPONSE_SLURM_RC) failed: Unexpected missing socket error Nov 05 08:50:47 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2268752700]] slurm_bufs_sendto(msg_type=RESPONSE_SLURM_RC) failed: Unexpected missing socket error Nov 05 08:50:47 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2268752701]] slurm_bufs_sendto(msg_type=RESPONSE_SLURM_RC) failed: Unexpected missing socket error Nov 05 08:50:47 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2268752704]] slurm_bufs_sendto(msg_type=RESPONSE_SLURM_RC) failed: Unexpected missing socket error Nov 05 08:50:47 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2268752704]] slurm_bufs_sendto(msg_type=RESPONSE_SLURM_RC) failed: Unexpected missing socket error Nov 05 08:50:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2824(8128414) Nov 05 08:50:49 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3454(8135310) Nov 05 08:50:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4602(8156178) Nov 05 08:50:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166579 NodeList=r911u05n03 #CPUs=36 Partition=pi_poland Nov 05 08:50:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:50:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2732(8166655) NodeList=r806u04n03 #CPUs=1 Partition=day Nov 05 08:50:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2733(8166656) NodeList=r909u10n03 #CPUs=1 Partition=day Nov 05 08:50:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2088(8164543) WEXITSTATUS 0 Nov 05 08:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2088(8164543) done Nov 05 08:50:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7285(8165961) WEXITSTATUS 1 Nov 05 08:51:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7285(8165961) done Nov 05 08:51:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:17 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 26141: QOSMaxSubmitJobPerUserLimit Nov 05 08:51:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 08:51:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6515(8164450) WEXITSTATUS 1 Nov 05 08:51:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6515(8164450) done Nov 05 08:51:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2183(8164836) WEXITSTATUS 0 Nov 05 08:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2183(8164836) done Nov 05 08:51:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1950(8163486) WEXITSTATUS 0 Nov 05 08:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1950(8163486) done Nov 05 08:51:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166531 uid 26141 Nov 05 08:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2241(8165024) WEXITSTATUS 0 Nov 05 08:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2241(8165024) done Nov 05 08:51:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:38 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166657 NodeList=(null) usec=16834 Nov 05 08:51:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166657 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 08:51:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3900(8166535) WEXITSTATUS 0 Nov 05 08:51:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3900(8166535) done Nov 05 08:51:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2734(8166658) NodeList=r918u32n01 #CPUs=1 Partition=day Nov 05 08:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2735(8166659) NodeList=r918u36n03 #CPUs=1 Partition=day Nov 05 08:51:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2736(8166660) NodeList=r815u23n02 #CPUs=1 Partition=day Nov 05 08:51:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1152(7573321) WEXITSTATUS 0 Nov 05 08:51:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1152(7573321) done Nov 05 08:51:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1906(8163137) WEXITSTATUS 0 Nov 05 08:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1906(8163137) done Nov 05 08:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4244(8152911) OOM failure Nov 05 08:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4244(8152911) done Nov 05 08:51:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:51:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166661 InitPrio=13510 usec=2496 Nov 05 08:52:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:52:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:52:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5201(8158564) WEXITSTATUS 1 Nov 05 08:52:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5201(8158564) done Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:52:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:52:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166662 InitPrio=25503 usec=4506 Nov 05 08:52:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:52:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_820(7572989) Nov 05 08:52:16 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2792(8128322) Nov 05 08:52:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:52:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:52:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3905(8166663) in gpu on r903u34n01 Nov 05 08:52:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3906(8166664) in gpu on r808u11n01 Nov 05 08:52:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2737(8166665) in day on r912u36n04 Nov 05 08:52:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166580 in gpu on r805u36n01 Nov 05 08:52:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166581 in gpu on r805u36n01 Nov 05 08:52:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166582 in gpu on r805u36n01 Nov 05 08:52:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:52:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6676680_6(6676680) in scavenge_gpu on r815u35n07 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7534(8166666) in scavenge on r815u15n09 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7535(8166667) in scavenge on r815u23n07 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7536(8166668) in scavenge on r816u23n12 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7537(8166669) in scavenge on r807u08n02 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7538(8166670) in scavenge on r909u13n04 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7539(8166671) in scavenge on r918u05n04 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7540(8166672) in scavenge on r919u09n02 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7541(8166673) in scavenge on r815u09n02 Nov 05 08:52:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7542(8166674) in scavenge on r815u09n02 Nov 05 08:52:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=81666661 uid 14266 Nov 05 08:52:46 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=81666661 Nov 05 08:52:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=14266 JobId=81666661 sig=9 returned: Invalid job id specified Nov 05 08:52:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166661 in scavenge on r906u01n02 Nov 05 08:52:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144166 NodeList=r912u05n03,r917u07n02 #CPUs=84 Partition=scavenge Nov 05 08:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2020(8164363) WEXITSTATUS 0 Nov 05 08:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2020(8164363) done Nov 05 08:52:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_830(8116304) WEXITSTATUS 0 Nov 05 08:52:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_830(8116304) done Nov 05 08:52:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166661 uid 14266 Nov 05 08:52:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166262 uid 25715 Nov 05 08:53:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3905(8166663) WEXITSTATUS 0 Nov 05 08:53:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3905(8166663) done Nov 05 08:53:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:01 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 08:53:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166272 uid 23627 Nov 05 08:53:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154701 WEXITSTATUS 0 Nov 05 08:53:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154701 done Nov 05 08:53:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166675 InitPrio=34694 usec=5310 Nov 05 08:53:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166675 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 08:53:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7107(8165674) WEXITSTATUS 1 Nov 05 08:53:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7107(8165674) done Nov 05 08:53:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3906(8166664) WEXITSTATUS 0 Nov 05 08:53:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3906(8166664) done Nov 05 08:53:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_39(8165295) WEXITSTATUS 0 Nov 05 08:53:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_39(8165295) done Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6227(8163408) WEXITSTATUS 1 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6227(8163408) done Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166676 InitPrio=13510 usec=2116 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166677 InitPrio=13510 usec=1664 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166678 InitPrio=13510 usec=2638 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166679 InitPrio=13510 usec=1841 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166680 InitPrio=13510 usec=2209 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166681 InitPrio=13510 usec=1583 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166682 InitPrio=13510 usec=1349 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166683 InitPrio=13510 usec=2024 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166684 InitPrio=13510 usec=1523 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166685 InitPrio=13510 usec=1346 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166686 InitPrio=13510 usec=1245 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166687 InitPrio=13510 usec=1248 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166688 InitPrio=13510 usec=1977 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166689 InitPrio=13510 usec=1748 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166690 InitPrio=13510 usec=1660 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166691 InitPrio=13510 usec=1477 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166692 InitPrio=13510 usec=1315 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166693 InitPrio=13510 usec=1261 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166694 InitPrio=13510 usec=1238 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166695 InitPrio=13510 usec=1250 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166696 InitPrio=13510 usec=1234 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166697 InitPrio=13510 usec=1192 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166698 InitPrio=13510 usec=1224 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166699 InitPrio=13510 usec=1253 Nov 05 08:53:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166700 InitPrio=13510 usec=1281 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166701 InitPrio=13510 usec=1591 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166702 InitPrio=13510 usec=1926 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166703 InitPrio=13510 usec=1485 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166704 InitPrio=13510 usec=1395 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166705 InitPrio=13510 usec=1292 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166706 InitPrio=13510 usec=1254 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166707 InitPrio=13510 usec=1293 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166708 InitPrio=13510 usec=1268 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166709 InitPrio=13510 usec=1221 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166710 InitPrio=13510 usec=1229 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166711 InitPrio=13510 usec=1277 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166712 InitPrio=13510 usec=1238 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166713 InitPrio=13510 usec=1200 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166714 InitPrio=13510 usec=1244 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166715 InitPrio=13510 usec=1232 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166716 InitPrio=13510 usec=1197 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166717 InitPrio=13510 usec=1234 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166718 InitPrio=13510 usec=1225 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166719 InitPrio=13510 usec=1196 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166720 InitPrio=13510 usec=1223 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166721 InitPrio=13510 usec=1286 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166722 InitPrio=13510 usec=1261 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166723 InitPrio=13510 usec=1210 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154676 WEXITSTATUS 0 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154676 done Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166724 InitPrio=13510 usec=1447 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166725 InitPrio=13510 usec=1585 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166726 InitPrio=13510 usec=1433 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166727 InitPrio=13510 usec=1272 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166728 InitPrio=13510 usec=1306 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166729 InitPrio=13510 usec=1243 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166730 InitPrio=13510 usec=1437 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166731 InitPrio=13510 usec=1296 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166732 InitPrio=13510 usec=1282 Nov 05 08:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166733 InitPrio=13509 usec=3509 Nov 05 08:53:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166662 NodeList=r913u01n03 #CPUs=24 Partition=day Nov 05 08:53:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7534(8166666) WEXITSTATUS 1 Nov 05 08:53:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7534(8166666) done Nov 05 08:53:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133196 WEXITSTATUS 0 Nov 05 08:53:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133196 done Nov 05 08:53:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7500(8166482) WEXITSTATUS 1 Nov 05 08:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7500(8166482) done Nov 05 08:53:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166734 InitPrio=34694 usec=4089 Nov 05 08:53:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166734 NodeList=r906u41n04 #CPUs=4 Partition=day Nov 05 08:53:46 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6966(8165286) has been requeued to reclaim resources for JobId=Invalid Nov 05 08:53:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6966(8165286) Nov 05 08:53:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:53:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154746 NodeList=r919u05n03 #CPUs=16 Partition=day Nov 05 08:53:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154747 NodeList=r816u09n06 #CPUs=16 Partition=day Nov 05 08:53:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166733 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 08:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7499(8166481) WEXITSTATUS 1 Nov 05 08:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7499(8166481) done Nov 05 08:53:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1962(8163594) WEXITSTATUS 0 Nov 05 08:53:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1962(8163594) done Nov 05 08:53:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166735 InitPrio=25503 usec=3892 Nov 05 08:53:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166736 InitPrio=20041 usec=4231 Nov 05 08:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166322 WEXITSTATUS 0 Nov 05 08:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166322 done Nov 05 08:53:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:53:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166735 NodeList=r815u23n05 #CPUs=24 Partition=day Nov 05 08:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907381 WEXITSTATUS 0 Nov 05 08:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907381 done Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:54:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:54:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:54:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6748(8164873) WEXITSTATUS 1 Nov 05 08:54:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6748(8164873) done Nov 05 08:54:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166733 WEXITSTATUS 0 Nov 05 08:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166733 done Nov 05 08:54:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:54:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7163(8165774) WEXITSTATUS 1 Nov 05 08:54:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7163(8165774) done Nov 05 08:54:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3898(8166503) WEXITSTATUS 0 Nov 05 08:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3898(8166503) done Nov 05 08:54:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_764(8115033) Nov 05 08:54:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2128(8125943) Nov 05 08:54:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2249(8126153) Nov 05 08:54:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2421(8126966) Nov 05 08:54:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2800(8128333) Nov 05 08:54:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:54:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 08:54:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:54:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166736 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u29n[02,04],r815u35n01,r816u15n03 #CPUs=288 Partition=day Nov 05 08:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1435(8135968) WEXITSTATUS 0 Nov 05 08:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1435(8135968) done Nov 05 08:54:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:55:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7239(8165896) WEXITSTATUS 1 Nov 05 08:55:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7239(8165896) done Nov 05 08:55:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:55:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6053(8163076) WEXITSTATUS 1 Nov 05 08:55:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6053(8163076) done Nov 05 08:55:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7522(8166541) WEXITSTATUS 1 Nov 05 08:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7522(8166541) done Nov 05 08:55:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:55:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3907(8166737) in gpu on r903u34n01 Nov 05 08:55:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3908(8166738) in gpu on r808u08n01 Nov 05 08:55:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3909(8166739) in gpu on r808u14n01 Nov 05 08:55:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907382 in gpu on r805u38n01 Nov 05 08:55:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2227(8164990) WEXITSTATUS 0 Nov 05 08:55:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2227(8164990) done Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7543(8166740) in scavenge on r802u06n04 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7544(8166741) in scavenge on r912u05n04 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7545(8166742) in scavenge on r917u01n02 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7546(8166743) in scavenge on r917u13n02 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7547(8166744) in scavenge on r815u15n09 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7548(8166745) in scavenge on r816u35n10 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7549(8166746) in scavenge on r4513u26n01 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7550(8166747) in scavenge on r806u04n02 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7551(8166748) in scavenge on r807u11n03 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7552(8166749) in scavenge on r909u01n03 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7553(8166750) in scavenge on r805u06n03 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7554(8166751) in scavenge on r805u11n01 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7555(8166752) in scavenge on r918u01n01 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7556(8166753) in scavenge on r919u05n03 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7557(8166754) in scavenge on r919u05n03 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_302(8114568) in scavenge on r816u15n08 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_249(7572417) in scavenge on r816u15n08 Nov 05 08:55:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_606(7572775) in scavenge on r816u23n03 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166676 in scavenge on r906u01n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166677 in scavenge on r907u01n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166678 in scavenge on r907u01n03 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166679 in scavenge on r907u01n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166680 in scavenge on r907u05n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166681 in scavenge on r907u05n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166682 in scavenge on r907u05n03 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166683 in scavenge on r907u05n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166684 in scavenge on r907u09n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166685 in scavenge on r917u13n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166686 in scavenge on r801u09n03 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166687 in scavenge on r801u09n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166688 in scavenge on r801u27n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166689 in scavenge on r802u14n03 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166690 in scavenge on r802u23n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166691 in scavenge on r802u30n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166692 in scavenge on r911u09n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166693 in scavenge on r911u09n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166694 in scavenge on r911u09n03 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166695 in scavenge on r911u13n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166696 in scavenge on r911u24n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166697 in scavenge on r911u28n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166698 in scavenge on r911u28n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166699 in scavenge on r912u05n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166700 in scavenge on r815u23n07 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166701 in scavenge on r815u29n09 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166702 in scavenge on r815u03n11 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166703 in scavenge on r815u03n12 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166704 in scavenge on r815u15n10 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166705 in scavenge on r816u23n11 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166706 in scavenge on r816u29n08 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166707 in scavenge on r816u29n11 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166708 in scavenge on r903u12n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166709 in scavenge on r911u32n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166710 in scavenge on r913u32n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166711 in scavenge on r913u35n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166712 in scavenge on r807u22n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166713 in scavenge on r4513u23n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166714 in scavenge on r4513u26n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166715 in scavenge on r902u07n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166716 in scavenge on r906u39n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166717 in scavenge on r815u35n07 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166718 in scavenge on r919u32n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166719 in scavenge on r919u32n03 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166720 in scavenge on r904u04n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166721 in scavenge on r904u04n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166722 in scavenge on r904u04n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166723 in scavenge on r904u10n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166724 in scavenge on r904u10n02 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166725 in scavenge on r904u10n03 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166726 in scavenge on r807u03n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166727 in scavenge on r807u05n01 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166728 in scavenge on r807u05n04 Nov 05 08:55:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166729 in scavenge on r807u08n01 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166730 in scavenge on r807u08n02 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166731 in scavenge on r807u11n01 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166732 in scavenge on r807u11n02 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2738(8166755) in scavenge on r908u28n03 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2739(8166756) in scavenge on r912u13n01 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2740(8166757) in scavenge on r4513u26n01 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2741(8166758) in scavenge on r815u35n07 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2742(8166759) in scavenge on r815u35n07 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2743(8166760) in scavenge on r815u35n07 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2744(8166761) in scavenge on r816u23n03 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2745(8166762) in scavenge on r816u23n03 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2746(8166763) in scavenge on r906u32n01 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2747(8166764) in scavenge on r805u32n01 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:134 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7552(8166749) WEXITSTATUS 1 Nov 05 08:55:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7552(8166749) done Nov 05 08:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166703 WEXITSTATUS 1 Nov 05 08:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166703 done Nov 05 08:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166713 WEXITSTATUS 1 Nov 05 08:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166713 done Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166715 WEXITSTATUS 1 Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166715 done Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166700 WEXITSTATUS 1 Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166700 done Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166704 WEXITSTATUS 1 Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166704 done Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166706 WEXITSTATUS 1 Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166706 done Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166714 WEXITSTATUS 1 Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166714 done Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166701 WEXITSTATUS 1 Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166701 done Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166702 WEXITSTATUS 1 Nov 05 08:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166702 done Nov 05 08:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166705 WEXITSTATUS 1 Nov 05 08:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166705 done Nov 05 08:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166707 WEXITSTATUS 1 Nov 05 08:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166707 done Nov 05 08:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166717 WEXITSTATUS 1 Nov 05 08:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166717 done Nov 05 08:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166676 WEXITSTATUS 1 Nov 05 08:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166676 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166684 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166684 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166719 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166719 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166682 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166682 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166678 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166678 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166683 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166683 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166681 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166681 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166677 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166677 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166718 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166718 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166689 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166689 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166725 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166725 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166679 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166679 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166687 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166687 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166712 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166712 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166729 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166729 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166731 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166731 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166730 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166730 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166720 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166720 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166724 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166724 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166732 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166732 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166688 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166688 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166709 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166709 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166685 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166685 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166728 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166728 done Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166708 WEXITSTATUS 1 Nov 05 08:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166708 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166695 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166695 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166694 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166694 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166686 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166686 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166723 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166723 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166721 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166721 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166690 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166690 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166727 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166727 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166699 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166699 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166716 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166716 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166726 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166726 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166680 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166680 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166710 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166710 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166722 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166722 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166711 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166711 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166693 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166693 done Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166698 WEXITSTATUS 1 Nov 05 08:55:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166698 done Nov 05 08:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166697 WEXITSTATUS 1 Nov 05 08:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166697 done Nov 05 08:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166692 WEXITSTATUS 1 Nov 05 08:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166692 done Nov 05 08:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166691 WEXITSTATUS 1 Nov 05 08:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166691 done Nov 05 08:55:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166696 WEXITSTATUS 1 Nov 05 08:55:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166696 done Nov 05 08:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2243(8165026) WEXITSTATUS 0 Nov 05 08:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2243(8165026) done Nov 05 08:55:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2748(8166765) NodeList=r815u03n08 #CPUs=1 Partition=day Nov 05 08:55:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2749(8166766) NodeList=r815u35n01 #CPUs=1 Partition=day Nov 05 08:55:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2160(8164765) WEXITSTATUS 0 Nov 05 08:55:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2160(8164765) done Nov 05 08:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7164(8165775) WEXITSTATUS 1 Nov 05 08:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7164(8165775) done Nov 05 08:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2209(8164934) WEXITSTATUS 0 Nov 05 08:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2209(8164934) done Nov 05 08:56:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2181(8164834) WEXITSTATUS 0 Nov 05 08:56:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2181(8164834) done Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:56:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7515(8166519) WEXITSTATUS 1 Nov 05 08:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7515(8166519) done Nov 05 08:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2187(8164854) WEXITSTATUS 0 Nov 05 08:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2187(8164854) done Nov 05 08:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7543(8166740) WEXITSTATUS 1 Nov 05 08:56:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7543(8166740) done Nov 05 08:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7349(8166058) WEXITSTATUS 1 Nov 05 08:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7349(8166058) done Nov 05 08:56:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6444(8164274) WEXITSTATUS 1 Nov 05 08:56:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6444(8164274) done Nov 05 08:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2750(8166767) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 08:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2751(8166768) NodeList=r909u01n03 #CPUs=1 Partition=day Nov 05 08:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2752(8166769) NodeList=r918u05n04 #CPUs=1 Partition=day Nov 05 08:56:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166770 InitPrio=57489 usec=4985 Nov 05 08:56:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166770 NodeList=r915u38n03 #CPUs=16 Partition=pi_lora Nov 05 08:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166770 WEXITSTATUS 0 Nov 05 08:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166770 done Nov 05 08:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7347(8166056) WEXITSTATUS 1 Nov 05 08:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7347(8166056) done Nov 05 08:57:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2204(8164906) WEXITSTATUS 0 Nov 05 08:57:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2204(8164906) done Nov 05 08:57:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7108(8165675) WEXITSTATUS 1 Nov 05 08:57:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7108(8165675) done Nov 05 08:57:26 r912u23n01 slurmctld[163760]: slurmctld: error: accounts_list_build: No association for account wv48-ct-hmis Nov 05 08:57:26 r912u23n01 slurmctld[163760]: error: accounts_list_build: No association for account wv48-ct-hmis Nov 05 08:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2036(8164426) WEXITSTATUS 0 Nov 05 08:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2036(8164426) done Nov 05 08:57:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3909(8166739) WEXITSTATUS 0 Nov 05 08:57:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3909(8166739) done Nov 05 08:57:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:57:49 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8074931_0(8074931) Nov 05 08:57:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:57:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2753(8166771) NodeList=r906u41n04 #CPUs=1 Partition=day Nov 05 08:57:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3910(8166772) in gpu on r808u14n01 Nov 05 08:58:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7558(8166773) in scavenge on r802u06n04 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7559(8166774) in scavenge on r802u11n04 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7560(8166775) in scavenge on r815u29n11 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7561(8166776) in scavenge on r808u17n01 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7562(8166777) in scavenge on r909u01n03 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7563(8166778) in scavenge on r913u01n01 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7564(8166779) in scavenge on r919u09n01 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7565(8166780) in scavenge on r805u14n01 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_710(7572879) in scavenge on r805u23n04 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159107 WEXITSTATUS 0 Nov 05 08:58:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159107 done Nov 05 08:58:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2754(8166781) in scavenge on r911u05n02 Nov 05 08:58:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2755(8166782) in scavenge on r815u35n07 Nov 05 08:58:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2756(8166783) in scavenge on r815u35n07 Nov 05 08:58:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2757(8166784) in scavenge on r906u28n01 Nov 05 08:58:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2638(8153025) WEXITSTATUS 0 Nov 05 08:58:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2638(8153025) done Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 08:58:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 08:58:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154697 WEXITSTATUS 0 Nov 05 08:58:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154697 done Nov 05 08:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6974(8165320) WEXITSTATUS 1 Nov 05 08:58:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6974(8165320) done Nov 05 08:58:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8154749 NodeList=r805u14n01 #CPUs=16 Partition=day Nov 05 08:58:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166785 InitPrio=17179 usec=6340 Nov 05 08:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1900(8163040) WEXITSTATUS 0 Nov 05 08:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1900(8163040) done Nov 05 08:59:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1998(8164174) WEXITSTATUS 0 Nov 05 08:59:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1998(8164174) done Nov 05 08:59:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 08:59:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3822(8165341) WEXITSTATUS 0 Nov 05 08:59:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3822(8165341) done Nov 05 08:59:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:59:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7228(8165876) WEXITSTATUS 1 Nov 05 08:59:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7228(8165876) done Nov 05 08:59:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:59:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1924(8163280) WEXITSTATUS 0 Nov 05 08:59:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1924(8163280) done Nov 05 08:59:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6829(8165016) WEXITSTATUS 1 Nov 05 08:59:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6829(8165016) done Nov 05 08:59:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:59:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3893(8166456) WEXITSTATUS 0 Nov 05 08:59:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3893(8166456) done Nov 05 08:59:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:59:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:59:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8074931_0(8074931) NodeList=r902u04n01 #CPUs=1 Partition=scavenge_gpu Nov 05 08:59:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_356(7572524) WEXITSTATUS 0 Nov 05 08:59:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_356(7572524) done Nov 05 08:59:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7338(8166042) WEXITSTATUS 1 Nov 05 08:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7338(8166042) done Nov 05 08:59:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 08:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7249(8165906) WEXITSTATUS 1 Nov 05 08:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7249(8165906) done Nov 05 08:59:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 09:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 09:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 09:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 09:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 09:00:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 09:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 09:00:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1917(8163251) WEXITSTATUS 0 Nov 05 09:00:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1917(8163251) done Nov 05 09:00:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:00:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2113(8164568) WEXITSTATUS 0 Nov 05 09:00:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2113(8164568) done Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:00:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:00:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166786 InitPrio=17179 usec=3835 Nov 05 09:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 09:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 09:00:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 09:00:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:00:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2246(8165038) WEXITSTATUS 0 Nov 05 09:00:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2246(8165038) done Nov 05 09:00:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166787 NodeList=(null) usec=17629 Nov 05 09:00:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166787 NodeList=r806u23n03 #CPUs=4 Partition=devel Nov 05 09:00:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3911(8166788) in gpu on r808u08n01 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3912(8166789) in gpu on r808u14n01 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166785 in day on r904u07n03 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166786 in day on r904u07n02 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2758(8166790) in day on r906u41n01 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2759(8166791) in day on r816u09n04 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2760(8166792) in day on r816u09n08 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2761(8166793) in day on r816u15n06 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2177(8164817) WEXITSTATUS 0 Nov 05 09:00:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2177(8164817) done Nov 05 09:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7566(8166794) in scavenge on r911u05n02 Nov 05 09:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7567(8166795) in scavenge on r4513u29n01 Nov 05 09:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7568(8166796) in scavenge on r906u28n01 Nov 05 09:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7569(8166797) in scavenge on r904u07n02 Nov 05 09:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7570(8166798) in scavenge on r904u07n02 Nov 05 09:00:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_331(8114598) in scavenge on r816u15n06 Nov 05 09:00:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6903(8165150) WEXITSTATUS 1 Nov 05 09:00:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6903(8165150) done Nov 05 09:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1951(8163487) WEXITSTATUS 0 Nov 05 09:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1951(8163487) done Nov 05 09:00:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8128368 Nov 05 09:00:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2762(8166799) NodeList=r815u23n02 #CPUs=1 Partition=day Nov 05 09:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2224(8164979) WEXITSTATUS 0 Nov 05 09:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2224(8164979) done Nov 05 09:00:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_6(8165333) WEXITSTATUS 0 Nov 05 09:00:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_6(8165333) done Nov 05 09:00:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2033(8164413) WEXITSTATUS 0 Nov 05 09:00:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2033(8164413) done Nov 05 09:01:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7539(8166671) WEXITSTATUS 1 Nov 05 09:01:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7539(8166671) done Nov 05 09:01:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166800 InitPrio=51301 usec=3534 Nov 05 09:01:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2211(8164946) WEXITSTATUS 0 Nov 05 09:01:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2211(8164946) done Nov 05 09:01:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6331(8163795) WEXITSTATUS 1 Nov 05 09:01:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6331(8163795) done Nov 05 09:01:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166786 uid 23242 Nov 05 09:01:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166801 InitPrio=51301 usec=4758 Nov 05 09:01:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158301 uid 12865 Nov 05 09:01:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6590(8164630) WEXITSTATUS 1 Nov 05 09:01:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6590(8164630) done Nov 05 09:01:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2763(8166802) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:01:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2764(8166803) NodeList=r918u09n03 #CPUs=1 Partition=day Nov 05 09:01:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2765(8166804) NodeList=r816u09n05 #CPUs=1 Partition=day Nov 05 09:01:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166805 InitPrio=57498 usec=4893 Nov 05 09:01:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166805 NodeList=r915u38n03 #CPUs=16 Partition=pi_lora Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:02:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:02:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1907(8163154) WEXITSTATUS 0 Nov 05 09:02:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1907(8163154) done Nov 05 09:02:15 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3555(8135865) has been requeued to reclaim resources for JobId=8166800 Nov 05 09:02:15 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8165203_16(8165252) had to be killed Nov 05 09:02:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3555(8135865) Nov 05 09:02:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6620(8164680) WEXITSTATUS 1 Nov 05 09:02:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6620(8164680) done Nov 05 09:02:19 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166800 NodeList=r911u01n03 #CPUs=36 Partition=pi_poland Nov 05 09:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 09:02:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 09:02:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 09:02:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5662(8162029) WEXITSTATUS 1 Nov 05 09:02:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5662(8162029) done Nov 05 09:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166806 InitPrio=38086 usec=7120 Nov 05 09:02:45 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166806 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5016(8158307) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5707(8162420) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6502(8164431) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2365(8165572) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2428(8165677) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2496(8165921) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2766(8166807) NodeList=r913u24n01 #CPUs=1 Partition=day Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7277(8165950) WEXITSTATUS 1 Nov 05 09:02:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7277(8165950) done Nov 05 09:02:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5707(8162420) Nov 05 09:02:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5016(8158307) Nov 05 09:02:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6502(8164431) Nov 05 09:02:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6676680_1(6676841) Nov 05 09:02:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2428(8165677) Nov 05 09:02:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2365(8165572) Nov 05 09:02:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2496(8165921) Nov 05 09:02:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166801 NodeList=r911u05n01 #CPUs=36 Partition=pi_poland Nov 05 09:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6944(8165236) WEXITSTATUS 1 Nov 05 09:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6944(8165236) done Nov 05 09:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2236(8165010) WEXITSTATUS 0 Nov 05 09:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2236(8165010) done Nov 05 09:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7533(8166576) WEXITSTATUS 1 Nov 05 09:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7533(8166576) done Nov 05 09:03:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2767(8166808) in day on r914u01n03 Nov 05 09:03:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7571(8166809) in scavenge on r802u02n02 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7572(8166810) in scavenge on r915u38n03 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7573(8166811) in scavenge on r4513u23n01 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7574(8166812) in scavenge on r914u01n03 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7575(8166813) in scavenge on r912u32n04 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7576(8166814) in scavenge on r913u24n01 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7577(8166815) in scavenge on r918u01n03 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7578(8166816) in scavenge on r918u05n03 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7579(8166817) in scavenge on r918u05n04 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7580(8166818) in scavenge on r805u02n01 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7581(8166819) in scavenge on r805u06n04 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7582(8166820) in scavenge on r805u23n02 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7583(8166821) in scavenge on r805u25n02 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_565(8114832) in scavenge on r815u35n05 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_109(7571073) in scavenge on r815u35n05 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6810(8164982) WEXITSTATUS 1 Nov 05 09:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6810(8164982) done Nov 05 09:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7378(8166125) WEXITSTATUS 1 Nov 05 09:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7378(8166125) done Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166822 InitPrio=10596 usec=3516 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166823 InitPrio=10596 usec=3326 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166824 InitPrio=10596 usec=2785 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166825 InitPrio=10596 usec=2642 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166826 InitPrio=10596 usec=2209 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166827 InitPrio=10596 usec=2117 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166828 InitPrio=10596 usec=2019 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166829 InitPrio=10596 usec=1951 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166830 InitPrio=10596 usec=2028 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166831 InitPrio=10596 usec=1950 Nov 05 09:03:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166832 InitPrio=10596 usec=2003 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166833 InitPrio=10596 usec=2220 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166834 InitPrio=10596 usec=2145 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166835 InitPrio=10596 usec=1905 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166836 InitPrio=10596 usec=1856 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166837 InitPrio=10596 usec=1953 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166838 InitPrio=10596 usec=1907 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166839 InitPrio=10596 usec=1920 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166840 InitPrio=10596 usec=2141 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166841 InitPrio=10596 usec=3807 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166842 InitPrio=10596 usec=3709 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166843 InitPrio=10596 usec=3205 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166844 InitPrio=10596 usec=3454 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166845 InitPrio=10596 usec=2357 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166846 InitPrio=10596 usec=2011 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166847 InitPrio=10596 usec=1980 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166848 InitPrio=10596 usec=2324 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166849 InitPrio=10596 usec=2121 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166850 InitPrio=10596 usec=1991 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166851 InitPrio=10596 usec=1926 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166852 InitPrio=10596 usec=1834 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166853 InitPrio=10596 usec=1756 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166854 InitPrio=10596 usec=2178 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166855 InitPrio=10596 usec=2153 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166856 InitPrio=10596 usec=2052 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166857 InitPrio=10596 usec=2362 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166858 InitPrio=10596 usec=1792 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166859 InitPrio=10596 usec=2111 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166860 InitPrio=10596 usec=2047 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166861 InitPrio=10596 usec=1908 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166862 InitPrio=10596 usec=3795 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166863 InitPrio=10596 usec=3040 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166864 InitPrio=10596 usec=2245 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166865 InitPrio=10596 usec=2289 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166866 InitPrio=10596 usec=2298 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166867 InitPrio=10596 usec=2391 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166868 InitPrio=10596 usec=3157 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166869 InitPrio=10596 usec=2323 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166870 InitPrio=10596 usec=2094 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166871 InitPrio=10596 usec=2871 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166872 InitPrio=10596 usec=2287 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166873 InitPrio=10596 usec=2164 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166874 InitPrio=10596 usec=2174 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166875 InitPrio=10596 usec=2143 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166876 InitPrio=10596 usec=2185 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166877 InitPrio=10596 usec=2130 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166878 InitPrio=10596 usec=2036 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166879 InitPrio=10596 usec=2045 Nov 05 09:03:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166880 InitPrio=10596 usec=1989 Nov 05 09:03:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2235(8165008) WEXITSTATUS 0 Nov 05 09:03:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2235(8165008) done Nov 05 09:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_61(8135760) WEXITSTATUS 0 Nov 05 09:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_61(8135760) done Nov 05 09:03:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1974(8163788) WEXITSTATUS 0 Nov 05 09:03:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1974(8163788) done Nov 05 09:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7576(8166814) WEXITSTATUS 1 Nov 05 09:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7576(8166814) done Nov 05 09:03:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166881 InitPrio=10596 usec=3443 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166882 InitPrio=10596 usec=2934 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166883 InitPrio=10596 usec=2188 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166884 InitPrio=10596 usec=3039 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166885 InitPrio=10596 usec=2667 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166886 InitPrio=10596 usec=2390 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166887 InitPrio=10596 usec=2119 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166888 InitPrio=10596 usec=3882 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166889 InitPrio=10596 usec=2598 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166890 InitPrio=10596 usec=2077 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166891 InitPrio=10596 usec=2997 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166892 InitPrio=10596 usec=2180 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166893 InitPrio=10596 usec=2004 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166894 InitPrio=10596 usec=2362 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166895 InitPrio=10596 usec=2026 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166896 InitPrio=10596 usec=1985 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166897 InitPrio=10596 usec=2008 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166898 InitPrio=10596 usec=2024 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166899 InitPrio=10596 usec=2300 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166900 InitPrio=10596 usec=2035 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166901 InitPrio=10596 usec=2159 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166902 InitPrio=10596 usec=2541 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166903 InitPrio=10596 usec=2205 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166904 InitPrio=10596 usec=3600 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166905 InitPrio=10596 usec=2950 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166906 InitPrio=10596 usec=2483 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166907 InitPrio=10596 usec=2267 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166908 InitPrio=10596 usec=2041 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166909 InitPrio=10596 usec=2560 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166910 InitPrio=10596 usec=2277 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166911 InitPrio=10596 usec=1900 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166912 InitPrio=10596 usec=1741 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166913 InitPrio=10596 usec=1768 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166914 InitPrio=10596 usec=2799 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166915 InitPrio=10596 usec=1810 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166916 InitPrio=10596 usec=3127 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166917 InitPrio=10596 usec=2744 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166918 InitPrio=10596 usec=1779 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166919 InitPrio=10596 usec=1678 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166920 InitPrio=10596 usec=1881 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166921 InitPrio=10596 usec=3005 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166922 InitPrio=10596 usec=2812 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166923 InitPrio=10596 usec=1933 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166924 InitPrio=10596 usec=2562 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166925 InitPrio=10596 usec=1889 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166926 InitPrio=10596 usec=1780 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166927 InitPrio=10596 usec=1801 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166928 InitPrio=10596 usec=2485 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166929 InitPrio=10596 usec=2291 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166930 InitPrio=10596 usec=2161 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166931 InitPrio=10596 usec=2153 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166932 InitPrio=10596 usec=1817 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166933 InitPrio=10596 usec=2433 Nov 05 09:03:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166934 InitPrio=10596 usec=2134 Nov 05 09:03:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166935 InitPrio=10596 usec=2781 Nov 05 09:03:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166936 InitPrio=10596 usec=4216 Nov 05 09:03:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166937 InitPrio=10596 usec=2367 Nov 05 09:03:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166938 InitPrio=10596 usec=3556 Nov 05 09:03:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166939 InitPrio=10596 usec=3935 Nov 05 09:03:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2768(8166940) NodeList=r918u32n04 #CPUs=1 Partition=day Nov 05 09:03:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2769(8166941) NodeList=r815u15n08 #CPUs=1 Partition=day Nov 05 09:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8166942 NodeList=(null) usec=22037 Nov 05 09:03:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166942 NodeList=r903u04n01 #CPUs=16 Partition=admintest Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166943 InitPrio=10596 usec=3360 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166944 InitPrio=10596 usec=4290 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166945 InitPrio=10596 usec=3728 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166946 InitPrio=10596 usec=4031 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166947 InitPrio=10596 usec=2581 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166948 InitPrio=10596 usec=3657 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166949 InitPrio=10596 usec=2974 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166950 InitPrio=10596 usec=2595 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166951 InitPrio=10596 usec=3188 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166952 InitPrio=10596 usec=3437 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166953 InitPrio=10596 usec=2634 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166954 InitPrio=10596 usec=2390 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166955 InitPrio=10596 usec=2360 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166956 InitPrio=10596 usec=3359 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166957 InitPrio=10596 usec=2531 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166958 InitPrio=10596 usec=2630 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166959 InitPrio=10596 usec=2502 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166960 InitPrio=10596 usec=2423 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166961 InitPrio=10596 usec=2324 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166962 InitPrio=10596 usec=3042 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166963 InitPrio=10596 usec=2210 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166964 InitPrio=10596 usec=2062 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166965 InitPrio=10596 usec=2215 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166966 InitPrio=10596 usec=3509 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166967 InitPrio=10596 usec=2941 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166968 InitPrio=10596 usec=2436 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166969 InitPrio=10596 usec=2309 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166970 InitPrio=10596 usec=2343 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166971 InitPrio=10596 usec=2246 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166972 InitPrio=10596 usec=2212 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166973 InitPrio=10596 usec=3198 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166974 InitPrio=10596 usec=2665 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166975 InitPrio=10596 usec=2247 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166976 InitPrio=10596 usec=2180 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166977 InitPrio=10596 usec=2073 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166978 InitPrio=10596 usec=2109 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166979 InitPrio=10596 usec=2178 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166980 InitPrio=10596 usec=2003 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166981 InitPrio=10596 usec=1884 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166982 InitPrio=10596 usec=1912 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166983 InitPrio=10596 usec=2446 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166984 InitPrio=10596 usec=2351 Nov 05 09:03:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166985 InitPrio=10596 usec=2270 Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166986 InitPrio=10596 usec=1961 Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166987 InitPrio=10596 usec=3274 Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166988 InitPrio=10596 usec=2995 Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166989 InitPrio=10596 usec=3511 Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:03:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:04:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:04:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3834(8165487) WEXITSTATUS 0 Nov 05 09:04:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3834(8165487) done Nov 05 09:04:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7329(8166026) WEXITSTATUS 1 Nov 05 09:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7329(8166026) done Nov 05 09:04:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1918(8163252) WEXITSTATUS 0 Nov 05 09:04:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1918(8163252) done Nov 05 09:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1987(8164043) WEXITSTATUS 0 Nov 05 09:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1987(8164043) done Nov 05 09:04:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:04:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2193(8164875) WEXITSTATUS 0 Nov 05 09:04:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2193(8164875) done Nov 05 09:04:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7562(8166777) WEXITSTATUS 1 Nov 05 09:04:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7562(8166777) done Nov 05 09:04:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2770(8166990) NodeList=r904u04n04 #CPUs=1 Partition=day Nov 05 09:04:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2771(8166991) NodeList=r815u15n07 #CPUs=1 Partition=day Nov 05 09:04:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1153(7572120) Nov 05 09:04:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2813(8128389) Nov 05 09:04:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2176(8164816) WEXITSTATUS 0 Nov 05 09:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2176(8164816) done Nov 05 09:04:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7546(8166743) WEXITSTATUS 1 Nov 05 09:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7546(8166743) done Nov 05 09:04:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1905(8163115) WEXITSTATUS 0 Nov 05 09:04:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1905(8163115) done Nov 05 09:04:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7548(8166745) WEXITSTATUS 1 Nov 05 09:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7548(8166745) done Nov 05 09:05:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7211(8165857) WEXITSTATUS 1 Nov 05 09:05:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7211(8165857) done Nov 05 09:05:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1964(8163640) WEXITSTATUS 0 Nov 05 09:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1964(8163640) done Nov 05 09:05:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2195(8164892) WEXITSTATUS 0 Nov 05 09:05:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2195(8164892) done Nov 05 09:05:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2225(8164980) WEXITSTATUS 0 Nov 05 09:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2225(8164980) done Nov 05 09:05:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8147672 uid 25502 Nov 05 09:05:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651306_0(7651309) Nov 05 09:05:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166992 InitPrio=21895 usec=3044 Nov 05 09:05:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7376(8166123) WEXITSTATUS 1 Nov 05 09:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7376(8166123) done Nov 05 09:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7348(8166057) WEXITSTATUS 1 Nov 05 09:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7348(8166057) done Nov 05 09:05:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166993 InitPrio=13574 usec=2408 Nov 05 09:05:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8147672 uid 25502 Nov 05 09:05:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2772(8166994) NodeList=r913u24n03 #CPUs=1 Partition=day Nov 05 09:05:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2773(8166995) NodeList=r914u05n03 #CPUs=1 Partition=day Nov 05 09:05:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2774(8166996) NodeList=r815u23n06 #CPUs=1 Partition=day Nov 05 09:05:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6676680_1(6676841) NodeList=r815u35n07 #CPUs=1 Partition=scavenge_gpu Nov 05 09:05:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8162707 Nov 05 09:05:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:05:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8151029_[2] uid 19099 Nov 05 09:05:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3913(8166997) in gpu on r808u17n01 Nov 05 09:06:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8166998 InitPrio=47165 usec=7726 Nov 05 09:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2061(8164516) WEXITSTATUS 0 Nov 05 09:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2061(8164516) done Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7584(8166999) in scavenge on r904u01n02 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7585(8167000) in scavenge on r802u23n02 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7586(8167001) in scavenge on r802u32n04 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7587(8167002) in scavenge on r917u13n02 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7588(8167003) in scavenge on r917u29n02 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7589(8167004) in scavenge on r815u03n12 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7590(8167005) in scavenge on r815u29n11 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7591(8167006) in scavenge on r816u35n10 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7592(8167007) in scavenge on r904u07n04 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7593(8167008) in scavenge on r909u01n03 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7594(8167009) in scavenge on r913u24n01 Nov 05 09:06:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_570(7571535) in scavenge on r816u29n05 Nov 05 09:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7202(8165842) WEXITSTATUS 1 Nov 05 09:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7202(8165842) done Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2775(8167010) in scavenge on r904u01n02 Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2776(8167011) in scavenge on r904u01n02 Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2777(8167012) in scavenge on r917u10n04 Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2778(8167013) in scavenge on r912u13n01 Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2779(8167014) in scavenge on r912u13n01 Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2780(8167015) in scavenge on r912u13n01 Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2781(8167016) in scavenge on r816u29n05 Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2782(8167017) in scavenge on r906u32n01 Nov 05 09:06:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2783(8167018) in scavenge on r805u36n01 Nov 05 09:06:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1429(8135962) WEXITSTATUS 0 Nov 05 09:06:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1429(8135962) done Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:06:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166993_[2] uid 19099 Nov 05 09:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2248(8165052) WEXITSTATUS 0 Nov 05 09:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2248(8165052) done Nov 05 09:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6561(8164583) WEXITSTATUS 1 Nov 05 09:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6561(8164583) done Nov 05 09:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2062(8164517) WEXITSTATUS 0 Nov 05 09:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2062(8164517) done Nov 05 09:06:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3640(8137391) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:06:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4185(8151444) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:06:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159210 has been requeued to reclaim resources for JobId=Invalid Nov 05 09:06:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6326(8163790) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:06:20 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2738(8166755) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:06:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3640(8137391) Nov 05 09:06:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6326(8163790) Nov 05 09:06:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159210 Nov 05 09:06:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4185(8151444) Nov 05 09:06:24 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2738(8166755) Nov 05 09:06:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167019 InitPrio=47164 usec=3988 Nov 05 09:06:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8166992 NodeList=r908u28n03 #CPUs=32 Partition=pi_seto Nov 05 09:06:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167020 InitPrio=13574 usec=3428 Nov 05 09:06:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166998 uid 21916 Nov 05 09:06:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7261(8165927) WEXITSTATUS 1 Nov 05 09:06:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7261(8165927) done Nov 05 09:06:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7314(8166002) WEXITSTATUS 1 Nov 05 09:06:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7314(8166002) done Nov 05 09:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7120(8165699) WEXITSTATUS 1 Nov 05 09:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7120(8165699) done Nov 05 09:06:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7506(8166498) WEXITSTATUS 1 Nov 05 09:06:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7506(8166498) done Nov 05 09:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1903(8163059) WEXITSTATUS 0 Nov 05 09:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1903(8163059) done Nov 05 09:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_175(8115645) WEXITSTATUS 0 Nov 05 09:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_175(8115645) done Nov 05 09:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1570(8149615) WEXITSTATUS 0 Nov 05 09:07:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1570(8149615) done Nov 05 09:07:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2192(8164874) WEXITSTATUS 0 Nov 05 09:07:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2192(8164874) done Nov 05 09:07:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167019 NodeList=r816u09n05 #CPUs=15 Partition=day Nov 05 09:07:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7138(8165730) WEXITSTATUS 1 Nov 05 09:07:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7138(8165730) done Nov 05 09:07:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3913(8166997) WEXITSTATUS 0 Nov 05 09:07:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3913(8166997) done Nov 05 09:07:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2063(8164518) WEXITSTATUS 0 Nov 05 09:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2063(8164518) done Nov 05 09:07:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:07:31 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8167021 NodeList=(null) usec=28172 Nov 05 09:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2226(8164989) WEXITSTATUS 0 Nov 05 09:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2226(8164989) done Nov 05 09:07:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167021 NodeList=r806u23n03 #CPUs=2 Partition=devel Nov 05 09:07:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:07:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7198(8165830) WEXITSTATUS 1 Nov 05 09:07:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7198(8165830) done Nov 05 09:07:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:07:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:07:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167022 InitPrio=27533 usec=4880 Nov 05 09:07:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:08:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161050 WEXITSTATUS 0 Nov 05 09:08:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8161050 done Nov 05 09:08:01 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167022 NodeList=r906u01n[01,03-04],r907u01n01,r907u09n[01,04] #CPUs=144 Partition=pi_fedorov Nov 05 09:08:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7557(8166754) WEXITSTATUS 1 Nov 05 09:08:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7557(8166754) done Nov 05 09:08:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6106(8163144) WEXITSTATUS 1 Nov 05 09:08:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6106(8163144) done Nov 05 09:08:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:08:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:08:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7531(8166574) WEXITSTATUS 1 Nov 05 09:08:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7531(8166574) done Nov 05 09:08:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2064(8164519) WEXITSTATUS 0 Nov 05 09:08:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2064(8164519) done Nov 05 09:08:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:08:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2212(8164958) WEXITSTATUS 0 Nov 05 09:08:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2212(8164958) done Nov 05 09:08:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:08:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7106(8165673) WEXITSTATUS 1 Nov 05 09:08:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7106(8165673) done Nov 05 09:08:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:08:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3914(8167023) in gpu on r808u17n01 Nov 05 09:08:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 09:08:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7651306_0(7651309) in gpu on r808u11n01 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7595(8167024) in scavenge on r917u29n03 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7596(8167025) in scavenge on r802u04n03 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7597(8167026) in scavenge on r802u11n03 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7598(8167027) in scavenge on r802u32n01 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7599(8167028) in scavenge on r815u29n08 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7600(8167029) in scavenge on r816u23n12 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7601(8167030) in scavenge on r907u28n01 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7602(8167031) in scavenge on r914u01n02 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7603(8167032) in scavenge on r919u05n03 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7604(8167033) in scavenge on r806u02n02 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7605(8167034) in scavenge on r806u23n01 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7606(8167035) in scavenge on r912u28n02 Nov 05 09:08:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167036 InitPrio=19191 usec=4124 Nov 05 09:08:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7607(8167037) in scavenge on r912u36n01 Nov 05 09:08:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7608(8167038) in scavenge on r815u09n07 Nov 05 09:08:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7609(8167039) in scavenge on r815u15n04 Nov 05 09:08:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159210 in scavenge on r906u01n02 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2784(8167040) in scavenge on r801u14n01 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2785(8167041) in scavenge on r815u35n09 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2786(8167042) in scavenge on r815u35n09 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2787(8167043) in scavenge on r815u35n09 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2788(8167044) in scavenge on r815u35n09 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2789(8167045) in scavenge on r912u09n01 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2790(8167046) in scavenge on r912u13n01 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2791(8167047) in scavenge on r906u39n01 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2792(8167048) in scavenge on r906u39n01 Nov 05 09:08:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2793(8167049) in scavenge on r906u32n01 Nov 05 09:08:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167036 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5013(8158304) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159207 has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159213 has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5566(8161220) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5568(8161222) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5570(8161224) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8163214 has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8163215 has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8163216 has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8163217 has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8163218 has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6212(8163364) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6258(8163540) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6484(8164400) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6555(8164577) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6783(8164937) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6916(8165176) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:50 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7141(8165736) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:08:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5570(8161224) Nov 05 09:08:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6916(8165176) Nov 05 09:08:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6555(8164577) Nov 05 09:08:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7141(8165736) Nov 05 09:08:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7366(8166104) WEXITSTATUS 1 Nov 05 09:08:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7366(8166104) done Nov 05 09:08:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8163218 Nov 05 09:08:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6484(8164400) Nov 05 09:08:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8163216 Nov 05 09:08:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6258(8163540) Nov 05 09:08:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159207 Nov 05 09:08:56 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5568(8161222) Nov 05 09:08:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5566(8161220) Nov 05 09:08:57 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6212(8163364) Nov 05 09:08:58 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5013(8158304) Nov 05 09:08:58 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8163217 Nov 05 09:08:59 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6783(8164937) Nov 05 09:08:59 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8163215 Nov 05 09:09:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159213 Nov 05 09:09:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8163214 Nov 05 09:09:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162657 WEXITSTATUS 0 Nov 05 09:09:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162657 done Nov 05 09:09:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7408(8166178) WEXITSTATUS 1 Nov 05 09:09:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7408(8166178) done Nov 05 09:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7411(8166190) WEXITSTATUS 1 Nov 05 09:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7411(8166190) done Nov 05 09:09:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2794(8167050) NodeList=r815u03n05 #CPUs=1 Partition=day Nov 05 09:09:52 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1914(8163234) WEXITSTATUS 0 Nov 05 09:10:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1914(8163234) done Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:10:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:10:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8162721 Nov 05 09:10:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7608(8167038) WEXITSTATUS 1 Nov 05 09:10:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7608(8167038) done Nov 05 09:10:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1971(8163742) WEXITSTATUS 0 Nov 05 09:10:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1971(8163742) done Nov 05 09:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2240(8165023) WEXITSTATUS 0 Nov 05 09:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2240(8165023) done Nov 05 09:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1975(8163789) WEXITSTATUS 0 Nov 05 09:10:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1975(8163789) done Nov 05 09:10:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2795(8167051) NodeList=r918u01n04 #CPUs=1 Partition=day Nov 05 09:10:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2796(8167052) NodeList=r919u32n02 #CPUs=1 Partition=day Nov 05 09:10:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2797(8167053) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:10:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2798(8167054) NodeList=r815u23n04 #CPUs=1 Partition=day Nov 05 09:10:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7508(8166500) WEXITSTATUS 1 Nov 05 09:10:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7508(8166500) done Nov 05 09:11:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4291(8154057) WEXITSTATUS 1 Nov 05 09:11:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4291(8154057) done Nov 05 09:11:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7581(8166819) WEXITSTATUS 1 Nov 05 09:11:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7581(8166819) done Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7610(8167055) in scavenge on r906u01n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7611(8167056) in scavenge on r906u01n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7612(8167057) in scavenge on r907u01n03 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7613(8167058) in scavenge on r907u01n03 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7614(8167059) in scavenge on r907u01n04 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7615(8167060) in scavenge on r907u01n04 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7616(8167061) in scavenge on r907u05n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7617(8167062) in scavenge on r907u05n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7618(8167063) in scavenge on r907u05n04 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7619(8167064) in scavenge on r907u05n04 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7620(8167065) in scavenge on r907u09n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7621(8167066) in scavenge on r907u09n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7622(8167067) in scavenge on r815u23n12 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7623(8167068) in scavenge on r919u36n01 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7624(8167069) in scavenge on r805u06n04 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7625(8167070) in scavenge on r805u25n01 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7626(8167071) in scavenge on r912u32n04 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7627(8167072) in scavenge on r913u13n01 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163218 in scavenge on r906u01n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163216 in scavenge on r906u01n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159207 in scavenge on r906u01n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163217 in scavenge on r801u30n01 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163215 in scavenge on r801u30n02 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159213 in scavenge on r801u30n04 Nov 05 09:11:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163214 in scavenge on r801u30n04 Nov 05 09:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7262(8165928) WEXITSTATUS 1 Nov 05 09:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7262(8165928) done Nov 05 09:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1915(8163249) WEXITSTATUS 0 Nov 05 09:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1915(8163249) done Nov 05 09:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2799(8167073) NodeList=r806u14n04 #CPUs=1 Partition=day Nov 05 09:11:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7266(8165932) WEXITSTATUS 1 Nov 05 09:11:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7266(8165932) done Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3903(8166538) WEXITSTATUS 0 Nov 05 09:12:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3903(8166538) done Nov 05 09:12:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:12:46 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8167074 NodeList=(null) usec=22461 Nov 05 09:12:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167074 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 09:12:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:12:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166675 uid 23627 Nov 05 09:13:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1935(8163361) WEXITSTATUS 0 Nov 05 09:13:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1935(8163361) done Nov 05 09:13:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167075 InitPrio=34706 usec=3570 Nov 05 09:13:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167075 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:13:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7541(8166673) WEXITSTATUS 1 Nov 05 09:13:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7541(8166673) done Nov 05 09:13:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7389(8166141) WEXITSTATUS 1 Nov 05 09:13:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7389(8166141) done Nov 05 09:13:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7404(8166174) WEXITSTATUS 1 Nov 05 09:13:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7404(8166174) done Nov 05 09:13:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7204(8165844) WEXITSTATUS 1 Nov 05 09:13:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7204(8165844) done Nov 05 09:13:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2800(8167076) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:13:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6812(8164984) WEXITSTATUS 1 Nov 05 09:13:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6812(8164984) done Nov 05 09:13:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7193(8165825) WEXITSTATUS 1 Nov 05 09:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7193(8165825) done Nov 05 09:13:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:14:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3915(8167077) in gpu on r4513u29n01 Nov 05 09:14:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 09:14:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 09:14:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7628(8167078) in scavenge on r917u13n02 Nov 05 09:14:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7629(8167079) in scavenge on r815u15n09 Nov 05 09:14:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7630(8167080) in scavenge on r815u29n12 Nov 05 09:14:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7631(8167081) in scavenge on r816u15n09 Nov 05 09:14:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7632(8167082) in scavenge on r806u06n01 Nov 05 09:14:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7633(8167083) in scavenge on r904u04n01 Nov 05 09:14:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7634(8167084) in scavenge on r913u01n02 Nov 05 09:14:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7635(8167085) in scavenge on r914u05n04 Nov 05 09:14:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:14:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3908(8166738) WEXITSTATUS 0 Nov 05 09:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3908(8166738) done Nov 05 09:14:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:14:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:15:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6688(8164790) WEXITSTATUS 1 Nov 05 09:15:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6688(8164790) done Nov 05 09:15:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:15:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7907002_3134(8128419) Nov 05 09:15:25 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:15:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:15:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7477(8166434) WEXITSTATUS 1 Nov 05 09:15:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7477(8166434) done Nov 05 09:15:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:15:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7170(8165788) WEXITSTATUS 1 Nov 05 09:15:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7170(8165788) done Nov 05 09:15:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7570(8166798) WEXITSTATUS 1 Nov 05 09:15:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7570(8166798) done Nov 05 09:15:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:15:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:16:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3902(8166537) WEXITSTATUS 0 Nov 05 09:16:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3902(8166537) done Nov 05 09:16:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_18(8165254) WEXITSTATUS 0 Nov 05 09:16:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_18(8165254) done Nov 05 09:16:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2106(8164561) WEXITSTATUS 0 Nov 05 09:16:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2106(8164561) done Nov 05 09:16:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:16:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1972(8163743) WEXITSTATUS 0 Nov 05 09:16:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1972(8163743) done Nov 05 09:16:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:16:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7145(8165740) WEXITSTATUS 1 Nov 05 09:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7145(8165740) done Nov 05 09:16:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154698 WEXITSTATUS 0 Nov 05 09:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154698 done Nov 05 09:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1943(8163403) WEXITSTATUS 0 Nov 05 09:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1943(8163403) done Nov 05 09:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7483(8166440) WEXITSTATUS 1 Nov 05 09:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7483(8166440) done Nov 05 09:16:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:16:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3916(8167086) in gpu on r808u08n01 Nov 05 09:16:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3917(8167087) in gpu on r808u11n01 Nov 05 09:16:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3918(8167088) in gpu on r4513u26n01 Nov 05 09:16:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154750 in day on r805u11n03 Nov 05 09:16:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2801(8167089) in day on r912u36n02 Nov 05 09:16:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2802(8167090) in day on r815u35n01 Nov 05 09:16:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7636(8167091) in scavenge on r916u04n01 Nov 05 09:16:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7637(8167092) in scavenge on r907u32n01 Nov 05 09:16:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7638(8167093) in scavenge on r904u07n02 Nov 05 09:16:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7639(8167094) in scavenge on r805u04n02 Nov 05 09:16:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7640(8167095) in scavenge on r805u25n04 Nov 05 09:16:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7641(8167096) in scavenge on r805u27n02 Nov 05 09:16:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2365(8165572) in scavenge on r806u31n01 Nov 05 09:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7144(8165739) WEXITSTATUS 1 Nov 05 09:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7144(8165739) done Nov 05 09:17:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167097 InitPrio=30393 usec=3276 Nov 05 09:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167097 NodeList=r805u40n01 #CPUs=8 Partition=bigmem Nov 05 09:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6808(8164978) WEXITSTATUS 1 Nov 05 09:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6808(8164978) done Nov 05 09:17:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1908(8163191) WEXITSTATUS 0 Nov 05 09:17:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1908(8163191) done Nov 05 09:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1911(8163231) WEXITSTATUS 0 Nov 05 09:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1911(8163231) done Nov 05 09:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_52(8135751) WEXITSTATUS 0 Nov 05 09:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_52(8135751) done Nov 05 09:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2803(8167098) NodeList=r806u06n01 #CPUs=1 Partition=day Nov 05 09:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2804(8167099) NodeList=r914u01n03 #CPUs=1 Partition=day Nov 05 09:17:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_259(7572427) Nov 05 09:17:53 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_587(7572756) Nov 05 09:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3917(8167087) WEXITSTATUS 0 Nov 05 09:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3917(8167087) done Nov 05 09:18:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7620(8167065) WEXITSTATUS 1 Nov 05 09:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7620(8167065) done Nov 05 09:18:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:18:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:18:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8167100 NodeList=(null) usec=32555 Nov 05 09:18:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167101 InitPrio=10821 usec=4407 Nov 05 09:18:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167102 InitPrio=10821 usec=3685 Nov 05 09:18:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160537 WEXITSTATUS 0 Nov 05 09:18:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160537 done Nov 05 09:18:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167103 InitPrio=74285 usec=8426 Nov 05 09:18:32 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167103 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:18:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167104 InitPrio=10821 usec=3437 Nov 05 09:18:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167105 InitPrio=10821 usec=3334 Nov 05 09:18:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2000(8164217) WEXITSTATUS 0 Nov 05 09:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2000(8164217) done Nov 05 09:18:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1986(8164042) WEXITSTATUS 0 Nov 05 09:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1986(8164042) done Nov 05 09:18:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167100 NodeList=r815u23n[01-04] #CPUs=4 Partition=day Nov 05 09:18:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2805(8167106) NodeList=r909u01n03 #CPUs=1 Partition=day Nov 05 09:18:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2806(8167107) NodeList=r815u15n04 #CPUs=1 Partition=day Nov 05 09:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1932(8163334) WEXITSTATUS 0 Nov 05 09:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1932(8163334) done Nov 05 09:18:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133195 WEXITSTATUS 0 Nov 05 09:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133195 done Nov 05 09:19:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:19:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7335(8166032) WEXITSTATUS 1 Nov 05 09:19:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7335(8166032) done Nov 05 09:19:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:19:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167101 in pi_ohern on r801u32n02 Nov 05 09:19:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167102 in pi_ohern on r801u32n02 Nov 05 09:19:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167104 in pi_ohern on r917u13n01 Nov 05 09:19:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167105 in pi_ohern on r917u13n01 Nov 05 09:19:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3919(8167108) in gpu on r808u11n01 Nov 05 09:19:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2807(8167109) in day on r805u04n03 Nov 05 09:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7390(8166142) WEXITSTATUS 1 Nov 05 09:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7390(8166142) done Nov 05 09:19:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 09:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154712 WEXITSTATUS 0 Nov 05 09:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154712 done Nov 05 09:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167102 WEXITSTATUS 1 Nov 05 09:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167102 done Nov 05 09:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167101 WEXITSTATUS 1 Nov 05 09:19:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167101 done Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7642(8167110) in scavenge on r907u09n02 Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7643(8167111) in scavenge on r917u13n03 Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7644(8167112) in scavenge on r907u34n01 Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7645(8167113) in scavenge on r914u05n03 Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7646(8167114) in scavenge on r914u05n03 Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_806(7571772) in scavenge on r816u35n01 Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167104 WEXITSTATUS 1 Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167104 done Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167105 WEXITSTATUS 1 Nov 05 09:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167105 done Nov 05 09:19:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 09:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7265(8165931) WEXITSTATUS 1 Nov 05 09:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7265(8165931) done Nov 05 09:19:37 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 09:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160344 WEXITSTATUS 0 Nov 05 09:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160344 done Nov 05 09:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1580(8151038) WEXITSTATUS 0 Nov 05 09:19:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1580(8151038) done Nov 05 09:19:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2808(8167115) NodeList=r816u09n05 #CPUs=1 Partition=day Nov 05 09:19:49 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8167116 NodeList=(null) usec=155596 Nov 05 09:19:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167116 NodeList=r816u15n09 #CPUs=1 Partition=pi_anticevic Nov 05 09:20:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1976(8163835) WEXITSTATUS 0 Nov 05 09:20:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1976(8163835) done Nov 05 09:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1431(8135964) WEXITSTATUS 0 Nov 05 09:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1431(8135964) done Nov 05 09:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3919(8167108) WEXITSTATUS 0 Nov 05 09:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3919(8167108) done Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167117 InitPrio=13509 usec=2077 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167118 InitPrio=13509 usec=1567 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167119 InitPrio=13509 usec=1493 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167120 InitPrio=13509 usec=2553 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167121 InitPrio=13509 usec=1662 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167122 InitPrio=13509 usec=1305 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167123 InitPrio=13509 usec=1293 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167124 InitPrio=13509 usec=1275 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167125 InitPrio=13509 usec=1239 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167126 InitPrio=13509 usec=1313 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167127 InitPrio=13509 usec=1246 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167128 InitPrio=13509 usec=1408 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167129 InitPrio=13509 usec=1777 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167130 InitPrio=13509 usec=1481 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167131 InitPrio=13509 usec=2491 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167132 InitPrio=13509 usec=1854 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167133 InitPrio=13509 usec=1428 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167134 InitPrio=13509 usec=1287 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167135 InitPrio=13509 usec=1339 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167136 InitPrio=13509 usec=1240 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167137 InitPrio=13509 usec=1273 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167138 InitPrio=13509 usec=1262 Nov 05 09:20:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167139 InitPrio=13509 usec=1262 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167140 InitPrio=13509 usec=1282 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167141 InitPrio=13509 usec=1322 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167142 InitPrio=13509 usec=1275 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167143 InitPrio=13509 usec=1261 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167144 InitPrio=13509 usec=1215 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167145 InitPrio=13509 usec=1672 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167146 InitPrio=13509 usec=1419 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167147 InitPrio=13509 usec=1230 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167148 InitPrio=13509 usec=1254 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167149 InitPrio=13509 usec=1237 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167150 InitPrio=13509 usec=1249 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167151 InitPrio=13509 usec=1246 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167152 InitPrio=13509 usec=1223 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167153 InitPrio=13509 usec=1218 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167154 InitPrio=13509 usec=1208 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167155 InitPrio=13509 usec=1219 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167156 InitPrio=13509 usec=1213 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167157 InitPrio=13509 usec=1229 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167158 InitPrio=13509 usec=1254 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167159 InitPrio=13509 usec=1251 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167160 InitPrio=13509 usec=1236 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167161 InitPrio=13509 usec=1201 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167162 InitPrio=13509 usec=1232 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167163 InitPrio=13509 usec=1254 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167164 InitPrio=13509 usec=2480 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167165 InitPrio=13509 usec=1806 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167166 InitPrio=13509 usec=1268 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167167 InitPrio=13509 usec=1246 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167168 InitPrio=13509 usec=1299 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167169 InitPrio=13509 usec=1666 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167170 InitPrio=13509 usec=1440 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167171 InitPrio=13509 usec=1257 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167172 InitPrio=13509 usec=1258 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167173 InitPrio=13509 usec=1239 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167174 InitPrio=13509 usec=1227 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167175 InitPrio=13509 usec=1283 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167176 InitPrio=13509 usec=1217 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167177 InitPrio=13509 usec=1221 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167178 InitPrio=13509 usec=1267 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167179 InitPrio=13509 usec=1467 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167180 InitPrio=13509 usec=1315 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167181 InitPrio=13509 usec=1247 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167182 InitPrio=13509 usec=1276 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167183 InitPrio=13509 usec=1256 Nov 05 09:20:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167184 InitPrio=13509 usec=1209 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167185 InitPrio=13509 usec=2601 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167186 InitPrio=13509 usec=2001 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167187 InitPrio=13509 usec=2167 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167188 InitPrio=13509 usec=1537 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167189 InitPrio=13509 usec=1591 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167190 InitPrio=13509 usec=1391 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167191 InitPrio=13509 usec=1436 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167192 InitPrio=13509 usec=1349 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167193 InitPrio=13509 usec=1282 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167194 InitPrio=13509 usec=1442 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167195 InitPrio=13509 usec=2363 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167196 InitPrio=13509 usec=1363 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167197 InitPrio=13509 usec=1703 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167198 InitPrio=13509 usec=2242 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167199 InitPrio=13509 usec=1628 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167200 InitPrio=13509 usec=1414 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167201 InitPrio=13509 usec=1656 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167202 InitPrio=13509 usec=1780 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167203 InitPrio=13509 usec=1589 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167204 InitPrio=13509 usec=1728 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167205 InitPrio=13509 usec=1383 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167206 InitPrio=13509 usec=1455 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167207 InitPrio=13509 usec=1615 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167208 InitPrio=13509 usec=2100 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167209 InitPrio=13509 usec=1653 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167210 InitPrio=13509 usec=1357 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167211 InitPrio=13509 usec=1649 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167212 InitPrio=13509 usec=1736 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167213 InitPrio=13509 usec=1409 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167214 InitPrio=13509 usec=1380 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167215 InitPrio=13509 usec=1298 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167216 InitPrio=13509 usec=1402 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167217 InitPrio=13509 usec=1653 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167218 InitPrio=13509 usec=1690 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167219 InitPrio=13509 usec=1423 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167220 InitPrio=13509 usec=1320 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167221 InitPrio=13509 usec=1622 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167222 InitPrio=13509 usec=1600 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167223 InitPrio=13509 usec=2238 Nov 05 09:20:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167224 InitPrio=13509 usec=1639 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167225 InitPrio=13509 usec=2043 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167226 InitPrio=13509 usec=2535 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167227 InitPrio=13509 usec=2022 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167228 InitPrio=13509 usec=2465 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167229 InitPrio=13509 usec=1585 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167230 InitPrio=13509 usec=1519 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167231 InitPrio=13509 usec=1785 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167232 InitPrio=13509 usec=1615 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167233 InitPrio=13509 usec=2012 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167234 InitPrio=13509 usec=1397 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167235 InitPrio=13509 usec=1673 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167236 InitPrio=13509 usec=1776 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167237 InitPrio=13509 usec=1559 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167238 InitPrio=13509 usec=1805 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167239 InitPrio=13509 usec=1638 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167240 InitPrio=13509 usec=1568 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167241 InitPrio=13509 usec=1567 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167242 InitPrio=13509 usec=1480 Nov 05 09:20:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167243 InitPrio=13509 usec=3667 Nov 05 09:20:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:20:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1993(8164131) WEXITSTATUS 0 Nov 05 09:20:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1993(8164131) done Nov 05 09:20:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:20:24 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2103(8125914) Nov 05 09:20:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2205(8164907) WEXITSTATUS 0 Nov 05 09:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2205(8164907) done Nov 05 09:20:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:20:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:20:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167243 NodeList=r906u41n04 #CPUs=1 Partition=day Nov 05 09:20:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2809(8167244) NodeList=r904u10n02 #CPUs=1 Partition=day Nov 05 09:20:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2810(8167245) NodeList=r918u09n01 #CPUs=1 Partition=day Nov 05 09:20:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2811(8167246) NodeList=r816u03n04 #CPUs=1 Partition=day Nov 05 09:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7584(8166999) WEXITSTATUS 1 Nov 05 09:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7584(8166999) done Nov 05 09:20:54 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:20:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7450(8166279) WEXITSTATUS 1 Nov 05 09:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7450(8166279) done Nov 05 09:21:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3911(8166788) WEXITSTATUS 0 Nov 05 09:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3911(8166788) done Nov 05 09:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1988(8164044) WEXITSTATUS 0 Nov 05 09:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1988(8164044) done Nov 05 09:21:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167247 InitPrio=21235 usec=3484 Nov 05 09:21:22 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167247 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:21:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7081(8165615) WEXITSTATUS 1 Nov 05 09:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7081(8165615) done Nov 05 09:21:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167243 WEXITSTATUS 0 Nov 05 09:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167243 done Nov 05 09:21:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7622(8167067) WEXITSTATUS 1 Nov 05 09:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7622(8167067) done Nov 05 09:21:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167248 InitPrio=33564 usec=5854 Nov 05 09:21:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7591(8167006) WEXITSTATUS 1 Nov 05 09:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7591(8167006) done Nov 05 09:21:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:21:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2812(8167249) NodeList=r815u15n07 #CPUs=1 Partition=day Nov 05 09:21:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167248 NodeList=r815u03n10 #CPUs=12 Partition=scavenge Nov 05 09:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7603(8167032) WEXITSTATUS 1 Nov 05 09:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7603(8167032) done Nov 05 09:21:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3920(8167250) in gpu on r808u08n01 Nov 05 09:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3921(8167251) in gpu on r808u11n01 Nov 05 09:22:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 09:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7647(8167252) in scavenge on r904u01n02 Nov 05 09:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7648(8167253) in scavenge on r815u03n10 Nov 05 09:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7649(8167254) in scavenge on r815u03n10 Nov 05 09:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7650(8167255) in scavenge on r815u23n12 Nov 05 09:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7651(8167256) in scavenge on r816u35n10 Nov 05 09:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7652(8167257) in scavenge on r907u28n01 Nov 05 09:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7653(8167258) in scavenge on r807u05n04 Nov 05 09:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7654(8167259) in scavenge on r918u05n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167117 in scavenge on r907u01n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167118 in scavenge on r907u01n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167119 in scavenge on r907u01n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167120 in scavenge on r907u05n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167121 in scavenge on r907u05n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167122 in scavenge on r907u05n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167123 in scavenge on r907u05n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167124 in scavenge on r907u09n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167125 in scavenge on r801u09n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167126 in scavenge on r801u09n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167127 in scavenge on r801u27n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167128 in scavenge on r802u14n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167129 in scavenge on r802u23n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167130 in scavenge on r802u30n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167131 in scavenge on r915u38n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167132 in scavenge on r917u13n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167133 in scavenge on r911u09n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167134 in scavenge on r911u09n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167135 in scavenge on r911u09n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167136 in scavenge on r911u13n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167137 in scavenge on r911u24n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167138 in scavenge on r911u28n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167139 in scavenge on r911u28n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167140 in scavenge on r912u05n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167141 in scavenge on r815u23n07 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167142 in scavenge on r815u29n09 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167143 in scavenge on r815u03n11 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167144 in scavenge on r815u03n12 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167145 in scavenge on r815u15n10 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167146 in scavenge on r816u23n11 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167147 in scavenge on r816u29n08 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167148 in scavenge on r816u29n11 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167149 in scavenge on r903u12n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167150 in scavenge on r911u32n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167151 in scavenge on r913u32n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167152 in scavenge on r913u35n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167153 in scavenge on r807u22n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167154 in scavenge on r4513u23n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167155 in scavenge on r4513u26n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167156 in scavenge on r902u07n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167157 in scavenge on r906u39n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167158 in scavenge on r906u41n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167159 in scavenge on r904u04n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167160 in scavenge on r904u04n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167161 in scavenge on r904u04n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167162 in scavenge on r904u10n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167163 in scavenge on r904u10n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167164 in scavenge on r904u10n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167165 in scavenge on r918u32n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167166 in scavenge on r919u32n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167167 in scavenge on r919u32n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167168 in scavenge on r807u03n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167169 in scavenge on r807u05n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167170 in scavenge on r807u05n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167171 in scavenge on r807u08n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167172 in scavenge on r807u08n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167173 in scavenge on r807u11n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167174 in scavenge on r807u11n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167175 in scavenge on r807u11n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167176 in scavenge on r806u02n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167177 in scavenge on r806u02n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167178 in scavenge on r806u04n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167179 in scavenge on r806u06n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167180 in scavenge on r806u09n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167181 in scavenge on r806u09n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167182 in scavenge on r806u11n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167183 in scavenge on r806u14n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167184 in scavenge on r806u14n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167185 in scavenge on r806u14n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167186 in scavenge on r912u28n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167187 in scavenge on r912u28n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167188 in scavenge on r912u28n04 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167189 in scavenge on r912u32n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167190 in scavenge on r912u32n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167191 in scavenge on r912u36n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167192 in scavenge on r913u01n01 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167193 in scavenge on r913u01n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167194 in scavenge on r913u09n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167195 in scavenge on r913u24n03 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167196 in scavenge on r913u28n02 Nov 05 09:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167197 in scavenge on r913u28n03 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167198 in scavenge on r913u28n04 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167199 in scavenge on r805u02n01 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167200 in scavenge on r805u02n02 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167201 in scavenge on r805u04n01 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167202 in scavenge on r805u04n02 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167203 in scavenge on r805u04n03 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167204 in scavenge on r805u06n04 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167205 in scavenge on r805u09n01 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167206 in scavenge on r805u09n02 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167207 in scavenge on r805u09n03 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167208 in scavenge on r805u14n02 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167209 in scavenge on r805u14n03 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167210 in scavenge on r805u14n04 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167211 in scavenge on r805u23n04 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167212 in scavenge on r805u25n01 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167213 in scavenge on r805u25n04 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167214 in scavenge on r805u27n02 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167215 in scavenge on r805u27n03 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167216 in scavenge on r815u09n03 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2813(8167260) in scavenge on r816u35n10 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2814(8167261) in scavenge on r805u32n01 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7444(8166273) WEXITSTATUS 1 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7444(8166273) done Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1947(8163443) WEXITSTATUS 0 Nov 05 09:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1947(8163443) done Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:22:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3920(8167250) WEXITSTATUS 0 Nov 05 09:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3920(8167250) done Nov 05 09:22:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167144 WEXITSTATUS 1 Nov 05 09:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167144 done Nov 05 09:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167154 WEXITSTATUS 1 Nov 05 09:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167154 done Nov 05 09:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167156 WEXITSTATUS 1 Nov 05 09:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167156 done Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167155 WEXITSTATUS 1 Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167155 done Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167141 WEXITSTATUS 1 Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167141 done Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167147 WEXITSTATUS 1 Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167147 done Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167145 WEXITSTATUS 1 Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167145 done Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167143 WEXITSTATUS 1 Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167143 done Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167148 WEXITSTATUS 1 Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167148 done Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167142 WEXITSTATUS 1 Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167142 done Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167146 WEXITSTATUS 1 Nov 05 09:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167146 done Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167121 WEXITSTATUS 1 Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167121 done Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1926(8163305) WEXITSTATUS 0 Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1926(8163305) done Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167118 WEXITSTATUS 1 Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167118 done Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167122 WEXITSTATUS 1 Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167122 done Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167123 WEXITSTATUS 1 Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167123 done Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167124 WEXITSTATUS 1 Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167124 done Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167117 WEXITSTATUS 1 Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167117 done Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167119 WEXITSTATUS 1 Nov 05 09:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167119 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167167 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167167 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167216 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167216 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167127 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167127 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167165 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167165 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167159 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167159 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167164 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167164 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167173 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167173 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167172 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167172 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167129 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167129 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167153 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167153 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167163 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167163 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167160 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167160 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167149 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167149 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167171 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167171 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167120 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167120 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167128 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167128 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167166 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167166 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167150 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167150 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167125 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167125 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167169 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167169 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167205 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167205 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167151 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167151 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167126 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167126 done Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167152 WEXITSTATUS 1 Nov 05 09:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167152 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167135 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167135 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167209 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167209 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167168 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167168 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167196 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167196 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167132 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167132 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167162 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167162 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167170 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167170 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167140 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167140 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167210 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167210 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167198 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167198 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167179 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167179 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167136 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167136 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167201 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167201 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167157 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167157 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167191 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167191 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167133 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167133 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167178 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167178 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167176 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167176 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167134 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167134 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167182 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167182 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167161 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167161 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167215 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167215 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167138 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167138 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167202 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167202 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167203 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167203 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167207 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167207 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167206 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167206 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167187 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167187 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167204 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167204 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167175 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167175 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167188 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167188 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167184 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167184 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167211 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167211 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167177 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167177 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167180 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167180 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167174 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167174 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167158 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167158 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167193 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167193 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167186 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167186 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167197 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167197 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167131 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167131 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167181 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167181 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167213 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167213 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167183 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167183 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167195 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167195 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167192 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167192 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167194 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167194 done Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167214 WEXITSTATUS 1 Nov 05 09:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167214 done Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167199 WEXITSTATUS 1 Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167199 done Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167208 WEXITSTATUS 1 Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167208 done Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167185 WEXITSTATUS 1 Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167185 done Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167212 WEXITSTATUS 1 Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167212 done Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167139 WEXITSTATUS 1 Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167139 done Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167200 WEXITSTATUS 1 Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167200 done Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167130 WEXITSTATUS 1 Nov 05 09:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167130 done Nov 05 09:22:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167137 WEXITSTATUS 1 Nov 05 09:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167137 done Nov 05 09:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167190 WEXITSTATUS 1 Nov 05 09:22:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167190 done Nov 05 09:22:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167189 WEXITSTATUS 1 Nov 05 09:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167189 done Nov 05 09:22:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7359(8166075) WEXITSTATUS 1 Nov 05 09:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7359(8166075) done Nov 05 09:22:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7597(8167026) WEXITSTATUS 1 Nov 05 09:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7597(8167026) done Nov 05 09:22:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167262 InitPrio=20039 usec=5173 Nov 05 09:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166736 WEXITSTATUS 0 Nov 05 09:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166736 done Nov 05 09:22:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:22:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167262 NodeList=r815u03n[02-03,08],r815u09n[01,03-04,08],r815u15n08,r815u29n[02,04],r815u35n01,r816u15n03 #CPUs=288 Partition=day Nov 05 09:22:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2815(8167263) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:22:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2816(8167264) NodeList=r918u09n02 #CPUs=1 Partition=day Nov 05 09:22:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2121(8164585) WEXITSTATUS 0 Nov 05 09:22:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2121(8164585) done Nov 05 09:23:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160534 WEXITSTATUS 0 Nov 05 09:23:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160534 done Nov 05 09:23:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2107(8164562) WEXITSTATUS 0 Nov 05 09:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2107(8164562) done Nov 05 09:23:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:23 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_305(7572473) Nov 05 09:23:23 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_343(7572511) Nov 05 09:23:23 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_562(7572731) Nov 05 09:23:23 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2838(8128458) Nov 05 09:23:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3916(8167086) WEXITSTATUS 0 Nov 05 09:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3916(8167086) done Nov 05 09:23:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7631(8167081) WEXITSTATUS 1 Nov 05 09:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7631(8167081) done Nov 05 09:23:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6149(8163196) WEXITSTATUS 1 Nov 05 09:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6149(8163196) done Nov 05 09:23:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1968(8163696) WEXITSTATUS 0 Nov 05 09:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1968(8163696) done Nov 05 09:23:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2817(8167265) NodeList=r807u08n03 #CPUs=1 Partition=day Nov 05 09:23:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2818(8167266) NodeList=r816u15n04 #CPUs=1 Partition=day Nov 05 09:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7637(8167092) WEXITSTATUS 1 Nov 05 09:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7637(8167092) done Nov 05 09:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1991(8164090) WEXITSTATUS 0 Nov 05 09:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1991(8164090) done Nov 05 09:23:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7298(8165978) WEXITSTATUS 1 Nov 05 09:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7298(8165978) done Nov 05 09:23:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:24:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7396(8166152) WEXITSTATUS 1 Nov 05 09:24:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7396(8166152) done Nov 05 09:24:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:24:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167267 InitPrio=23568 usec=5237 Nov 05 09:24:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:24:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3424(8135254) WEXITSTATUS 1 Nov 05 09:24:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3424(8135254) done Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:24:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:24:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166159 WEXITSTATUS 0 Nov 05 09:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166159 done Nov 05 09:24:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_0(8167268) in day on r906u41n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_1(8167269) in day on r906u41n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_2(8167270) in day on r904u07n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_3(8167271) in day on r904u07n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_4(8167272) in day on r904u07n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_5(8167273) in day on r904u07n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_6(8167274) in day on r904u07n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_7(8167275) in day on r904u07n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_8(8167276) in day on r904u07n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_9(8167277) in day on r904u07n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_10(8167278) in day on r904u07n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_11(8167279) in day on r904u07n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_12(8167280) in day on r906u41n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_13(8167281) in day on r906u41n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_14(8167282) in day on r906u41n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_15(8167283) in day on r906u41n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_16(8167284) in day on r906u41n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_17(8167285) in day on r904u04n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_18(8167286) in day on r904u04n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_19(8167287) in day on r904u04n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_20(8167288) in day on r904u04n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_21(8167289) in day on r904u04n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_22(8167290) in day on r904u04n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_23(8167291) in day on r904u04n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_24(8167292) in day on r904u04n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_25(8167293) in day on r904u04n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_26(8167294) in day on r904u04n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_27(8167295) in day on r904u04n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_28(8167296) in day on r904u04n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_29(8167297) in day on r904u04n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_30(8167298) in day on r904u04n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_31(8167299) in day on r904u04n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_32(8167300) in day on r904u04n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_33(8167301) in day on r904u04n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_34(8167302) in day on r904u04n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_35(8167303) in day on r904u04n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_36(8167304) in day on r904u04n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_37(8167305) in day on r904u10n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_38(8167306) in day on r904u10n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_39(8167307) in day on r904u10n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_40(8167308) in day on r904u10n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_41(8167309) in day on r904u10n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_42(8167310) in day on r904u10n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_43(8167311) in day on r904u10n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_44(8167312) in day on r904u10n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_45(8167313) in day on r904u10n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_46(8167314) in day on r904u10n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_47(8167315) in day on r904u10n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_48(8167316) in day on r904u10n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_49(8167317) in day on r904u10n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_50(8167318) in day on r904u10n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_51(8167319) in day on r904u10n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_52(8167320) in day on r914u01n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_53(8167321) in day on r914u01n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_54(8167322) in day on r914u01n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_55(8167323) in day on r914u01n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_56(8167324) in day on r914u01n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_57(8167325) in day on r914u01n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_58(8167326) in day on r914u01n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_59(8167327) in day on r914u01n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_60(8167328) in day on r914u01n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_61(8167329) in day on r914u01n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_62(8167330) in day on r914u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_63(8167331) in day on r914u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_64(8167332) in day on r914u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_65(8167333) in day on r914u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_66(8167334) in day on r914u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_67(8167335) in day on r914u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_68(8167336) in day on r914u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_69(8167337) in day on r914u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_70(8167338) in day on r914u05n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_71(8167339) in day on r914u05n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_72(8167340) in day on r914u05n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_73(8167341) in day on r914u05n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_74(8167342) in day on r914u05n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_75(8167343) in day on r914u05n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_76(8167344) in day on r807u03n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_77(8167345) in day on r807u03n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_78(8167346) in day on r807u03n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_79(8167347) in day on r807u03n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_80(8167348) in day on r807u03n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_81(8167349) in day on r807u03n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_82(8167350) in day on r807u03n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_83(8167351) in day on r807u03n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_84(8167352) in day on r807u03n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_85(8167353) in day on r807u03n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_86(8167354) in day on r807u03n04 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_87(8167355) in day on r807u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_88(8167356) in day on r807u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_89(8167357) in day on r807u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_90(8167358) in day on r807u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_91(8167359) in day on r807u05n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_92(8167360) in day on r807u05n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_93(8167361) in day on r807u05n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_94(8167362) in day on r807u05n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_95(8167363) in day on r807u05n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_96(8167364) in day on r807u05n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_97(8167365) in day on r807u05n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_98(8167366) in day on r807u05n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167267_99(8167367) in day on r807u05n03 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3922(8167368) in gpu on r808u08n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3923(8167369) in gpu on r808u08n01 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2819(8167370) in day on r815u35n02 Nov 05 09:24:46 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:206 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 09:24:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u05n03 Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_100(8167371) NodeList=r807u05n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_101(8167372) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_102(8167373) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_103(8167374) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_104(8167375) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_105(8167376) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_106(8167377) NodeList=r807u08n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_107(8167378) NodeList=r807u08n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_108(8167379) NodeList=r807u08n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_109(8167380) NodeList=r807u08n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_110(8167381) NodeList=r807u08n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_111(8167382) NodeList=r807u08n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_112(8167383) NodeList=r807u08n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_113(8167384) NodeList=r807u08n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_114(8167385) NodeList=r807u08n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_115(8167386) NodeList=r807u08n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_116(8167387) NodeList=r807u08n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_117(8167388) NodeList=r807u08n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_118(8167389) NodeList=r807u08n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_119(8167390) NodeList=r807u08n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_120(8167391) NodeList=r807u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_121(8167392) NodeList=r807u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_122(8167393) NodeList=r807u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_123(8167394) NodeList=r807u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_124(8167395) NodeList=r807u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_125(8167396) NodeList=r807u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_126(8167397) NodeList=r807u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_127(8167398) NodeList=r807u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_128(8167399) NodeList=r807u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_129(8167400) NodeList=r807u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_130(8167401) NodeList=r807u11n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_131(8167402) NodeList=r807u11n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_132(8167403) NodeList=r807u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_133(8167404) NodeList=r807u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_134(8167405) NodeList=r807u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_135(8167406) NodeList=r807u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_136(8167407) NodeList=r807u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_137(8167408) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_138(8167409) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_139(8167410) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_140(8167411) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_141(8167412) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_142(8167413) NodeList=r909u01n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_143(8167414) NodeList=r909u01n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_144(8167415) NodeList=r909u01n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_145(8167416) NodeList=r909u01n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_146(8167417) NodeList=r909u01n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_147(8167418) NodeList=r909u01n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_148(8167419) NodeList=r909u01n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_149(8167420) NodeList=r909u01n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_150(8167421) NodeList=r909u01n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_151(8167422) NodeList=r909u01n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_152(8167423) NodeList=r909u04n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_153(8167424) NodeList=r909u04n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_154(8167425) NodeList=r909u04n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_155(8167426) NodeList=r909u04n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_156(8167427) NodeList=r909u04n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_157(8167428) NodeList=r909u04n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_158(8167429) NodeList=r909u07n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_159(8167430) NodeList=r909u07n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_160(8167431) NodeList=r909u07n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_161(8167432) NodeList=r909u07n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_162(8167433) NodeList=r909u07n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_163(8167434) NodeList=r909u07n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_164(8167435) NodeList=r909u07n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_165(8167436) NodeList=r909u07n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_166(8167437) NodeList=r909u10n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_167(8167438) NodeList=r909u10n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_168(8167439) NodeList=r909u10n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_169(8167440) NodeList=r909u10n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_170(8167441) NodeList=r909u10n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_171(8167442) NodeList=r909u10n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_172(8167443) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_173(8167444) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_174(8167445) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_175(8167446) NodeList=r909u13n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_176(8167447) NodeList=r909u13n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_177(8167448) NodeList=r909u13n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_178(8167449) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_179(8167450) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_180(8167451) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_181(8167452) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_182(8167453) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_183(8167454) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_184(8167455) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_185(8167456) NodeList=r909u13n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_186(8167457) NodeList=r909u13n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_187(8167458) NodeList=r909u13n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_188(8167459) NodeList=r909u13n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_189(8167460) NodeList=r909u29n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_190(8167461) NodeList=r909u29n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_191(8167462) NodeList=r909u29n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_192(8167463) NodeList=r909u29n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_193(8167464) NodeList=r909u29n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_194(8167465) NodeList=r806u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_195(8167466) NodeList=r806u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_196(8167467) NodeList=r806u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_197(8167468) NodeList=r806u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_198(8167469) NodeList=r806u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_199(8167470) NodeList=r806u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_200(8167471) NodeList=r806u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_201(8167472) NodeList=r806u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_202(8167473) NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_203(8167474) NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_204(8167475) NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_205(8167476) NodeList=r806u02n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_206(8167477) NodeList=r806u02n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_207(8167478) NodeList=r806u02n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_208(8167479) NodeList=r806u02n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_209(8167480) NodeList=r806u02n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_210(8167481) NodeList=r806u02n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_211(8167482) NodeList=r806u02n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_212(8167483) NodeList=r806u02n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_213(8167484) NodeList=r806u02n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_214(8167485) NodeList=r806u04n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_215(8167486) NodeList=r806u04n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_216(8167487) NodeList=r806u04n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_217(8167488) NodeList=r806u04n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_218(8167489) NodeList=r806u04n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_219(8167490) NodeList=r806u04n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_220(8167491) NodeList=r806u04n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_221(8167492) NodeList=r806u04n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_222(8167493) NodeList=r806u04n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_223(8167494) NodeList=r806u04n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_224(8167495) NodeList=r806u04n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_225(8167496) NodeList=r806u04n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_226(8167497) NodeList=r806u06n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_227(8167498) NodeList=r806u06n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_228(8167499) NodeList=r806u06n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_229(8167500) NodeList=r806u06n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_230(8167501) NodeList=r806u06n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_231(8167502) NodeList=r806u06n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_232(8167503) NodeList=r806u06n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_233(8167504) NodeList=r806u06n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_234(8167505) NodeList=r806u06n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_235(8167506) NodeList=r806u06n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_236(8167507) NodeList=r806u06n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_237(8167508) NodeList=r806u06n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_238(8167509) NodeList=r806u06n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_239(8167510) NodeList=r806u06n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_240(8167511) NodeList=r806u06n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_241(8167512) NodeList=r806u06n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_242(8167513) NodeList=r806u06n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_243(8167514) NodeList=r806u06n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_244(8167515) NodeList=r806u09n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_245(8167516) NodeList=r806u09n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_246(8167517) NodeList=r806u09n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_247(8167518) NodeList=r806u09n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_248(8167519) NodeList=r806u09n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_249(8167520) NodeList=r806u09n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_250(8167521) NodeList=r806u09n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_251(8167522) NodeList=r806u09n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_252(8167523) NodeList=r806u09n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_253(8167524) NodeList=r806u09n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_254(8167525) NodeList=r806u09n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_255(8167526) NodeList=r806u09n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_256(8167527) NodeList=r806u09n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_257(8167528) NodeList=r806u09n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_258(8167529) NodeList=r806u09n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_259(8167530) NodeList=r806u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_260(8167531) NodeList=r806u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_261(8167532) NodeList=r806u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_262(8167533) NodeList=r806u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_263(8167534) NodeList=r806u11n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_264(8167535) NodeList=r806u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_265(8167536) NodeList=r806u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_266(8167537) NodeList=r806u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_267(8167538) NodeList=r806u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_268(8167539) NodeList=r806u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_269(8167540) NodeList=r806u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_270(8167541) NodeList=r806u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_271(8167542) NodeList=r806u11n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_272(8167543) NodeList=r806u11n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_273(8167544) NodeList=r806u11n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_274(8167545) NodeList=r806u11n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_275(8167546) NodeList=r806u11n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_276(8167547) NodeList=r806u11n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_277(8167548) NodeList=r806u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_278(8167549) NodeList=r806u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_279(8167550) NodeList=r806u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_280(8167551) NodeList=r806u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_281(8167552) NodeList=r806u11n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_282(8167553) NodeList=r806u14n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_283(8167554) NodeList=r806u14n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_284(8167555) NodeList=r806u14n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_285(8167556) NodeList=r806u14n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_286(8167557) NodeList=r806u14n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_287(8167558) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_288(8167559) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_289(8167560) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_290(8167561) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_291(8167562) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_292(8167563) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_293(8167564) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_294(8167565) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_295(8167566) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_296(8167567) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_297(8167568) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_298(8167569) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_299(8167570) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_300(8167571) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_301(8167572) NodeList=r806u14n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_302(8167573) NodeList=r806u14n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_303(8167574) NodeList=r806u14n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_304(8167575) NodeList=r806u14n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_305(8167576) NodeList=r806u14n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_306(8167577) NodeList=r806u14n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_307(8167578) NodeList=r806u14n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_308(8167579) NodeList=r806u14n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_309(8167580) NodeList=r806u14n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_310(8167581) NodeList=r806u14n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_311(8167582) NodeList=r806u14n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_312(8167583) NodeList=r806u23n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_313(8167584) NodeList=r806u23n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_314(8167585) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_315(8167586) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_316(8167587) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_317(8167588) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_318(8167589) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_319(8167590) NodeList=r805u02n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_320(8167591) NodeList=r805u02n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_321(8167592) NodeList=r805u02n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_322(8167593) NodeList=r805u02n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_323(8167594) NodeList=r805u02n02 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_324(8167595) NodeList=r805u02n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_325(8167596) NodeList=r805u02n03 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_326(8167597) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:24:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_327(8167598) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167599 InitPrio=33564 usec=2427 Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167600 InitPrio=33564 usec=1949 Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167601 InitPrio=33564 usec=1581 Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167602 InitPrio=33564 usec=1168 Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167603 InitPrio=33564 usec=1036 Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167604 InitPrio=33564 usec=1864 Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167605 InitPrio=33564 usec=1216 Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167606 InitPrio=33564 usec=1252 Nov 05 09:24:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166942 WEXITSTATUS 0 Nov 05 09:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166942 done Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7655(8167607) in scavenge on r801u14n01 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7656(8167608) in scavenge on r802u11n03 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7657(8167609) in scavenge on r917u29n01 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7658(8167610) in scavenge on r815u15n09 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7659(8167611) in scavenge on r816u29n08 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7660(8167612) in scavenge on r816u15n09 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7661(8167613) in scavenge on r816u15n12 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7662(8167614) in scavenge on r907u32n01 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7663(8167615) in scavenge on r805u06n04 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7664(8167616) in scavenge on r918u32n04 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1007(7571973) in scavenge on r801u06n02 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1015(7571981) in scavenge on r802u04n01 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_435(7572603) in scavenge on r802u04n03 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_579(7572748) in scavenge on r816u35n05 Nov 05 09:24:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_696(7572865) in scavenge on r904u13n01 Nov 05 09:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6048(8163071) WEXITSTATUS 1 Nov 05 09:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6048(8163071) done Nov 05 09:24:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_328(8167617) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167217 in scavenge on r907u01n02 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167218 in scavenge on r907u01n03 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167219 in scavenge on r907u01n04 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167220 in scavenge on r907u05n01 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167221 in scavenge on r907u05n02 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167222 in scavenge on r907u05n03 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167223 in scavenge on r907u05n04 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167224 in scavenge on r907u09n02 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167225 in scavenge on r801u09n03 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167226 in scavenge on r801u09n04 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167227 in scavenge on r801u27n02 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167228 in scavenge on r802u14n03 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167229 in scavenge on r802u23n02 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167230 in scavenge on r802u30n04 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167231 in scavenge on r915u38n03 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167232 in scavenge on r917u13n04 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167233 in scavenge on r911u09n01 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167234 in scavenge on r911u09n02 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167235 in scavenge on r911u09n03 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167236 in scavenge on r911u13n01 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167237 in scavenge on r911u24n02 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167238 in scavenge on r911u28n01 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167239 in scavenge on r911u28n02 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167240 in scavenge on r912u05n04 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167241 in scavenge on r815u23n07 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167242 in scavenge on r815u29n09 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2428(8165677) in scavenge on r816u35n05 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2496(8165921) in scavenge on r816u35n05 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2738(8166755) in scavenge on r816u35n05 Nov 05 09:24:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_329(8167618) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:25:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8167619 NodeList=(null) usec=21760 Nov 05 09:25:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167619 NodeList=r801u23n02 #CPUs=32 Partition=admintest Nov 05 09:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7432(8166225) WEXITSTATUS 1 Nov 05 09:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7432(8166225) done Nov 05 09:25:09 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_330(8167620) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:25:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167241 WEXITSTATUS 1 Nov 05 09:25:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167241 done Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_331(8167621) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167242 WEXITSTATUS 1 Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167242 done Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167221 WEXITSTATUS 1 Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167221 done Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167223 WEXITSTATUS 1 Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167223 done Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167218 WEXITSTATUS 1 Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167218 done Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167222 WEXITSTATUS 1 Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167222 done Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167224 WEXITSTATUS 1 Nov 05 09:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167224 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167217 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167217 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167219 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167219 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167229 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167229 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167228 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167228 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167220 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167220 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167225 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167225 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167226 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167226 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167232 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167232 done Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167235 WEXITSTATUS 1 Nov 05 09:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167235 done Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_332(8167622) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167231 WEXITSTATUS 1 Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167231 done Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167227 WEXITSTATUS 1 Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167227 done Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167240 WEXITSTATUS 1 Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167240 done Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167233 WEXITSTATUS 1 Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167233 done Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167230 WEXITSTATUS 1 Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167230 done Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167234 WEXITSTATUS 1 Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167234 done Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167236 WEXITSTATUS 1 Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167236 done Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167238 WEXITSTATUS 1 Nov 05 09:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167238 done Nov 05 09:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167239 WEXITSTATUS 1 Nov 05 09:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167239 done Nov 05 09:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167237 WEXITSTATUS 1 Nov 05 09:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167237 done Nov 05 09:25:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_333(8167623) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3910(8166772) WEXITSTATUS 0 Nov 05 09:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3910(8166772) done Nov 05 09:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7580(8166818) WEXITSTATUS 1 Nov 05 09:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7580(8166818) done Nov 05 09:25:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1928(8163307) WEXITSTATUS 0 Nov 05 09:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1928(8163307) done Nov 05 09:25:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4903(8158036) WEXITSTATUS 1 Nov 05 09:25:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4903(8158036) done Nov 05 09:25:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7554(8166751) WEXITSTATUS 1 Nov 05 09:25:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7554(8166751) done Nov 05 09:25:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7124(8165707) WEXITSTATUS 1 Nov 05 09:25:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7124(8165707) done Nov 05 09:25:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7340(8166044) WEXITSTATUS 1 Nov 05 09:25:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7340(8166044) done Nov 05 09:25:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6996(8165390) WEXITSTATUS 1 Nov 05 09:25:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6996(8165390) done Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_334(8167624) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_335(8167625) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_336(8167626) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_337(8167627) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_338(8167628) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_339(8167629) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_340(8167630) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_341(8167631) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_342(8167632) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_343(8167633) NodeList=r805u02n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_344(8167634) NodeList=r805u02n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_345(8167635) NodeList=r805u04n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_346(8167636) NodeList=r805u04n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_347(8167637) NodeList=r805u04n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_348(8167638) NodeList=r805u04n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_349(8167639) NodeList=r805u04n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_350(8167640) NodeList=r805u04n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_351(8167641) NodeList=r805u04n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_352(8167642) NodeList=r805u04n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_353(8167643) NodeList=r805u04n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_354(8167644) NodeList=r805u04n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_355(8167645) NodeList=r805u04n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_356(8167646) NodeList=r805u04n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_357(8167647) NodeList=r805u04n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_358(8167648) NodeList=r805u04n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_359(8167649) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_360(8167650) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_361(8167651) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_362(8167652) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_363(8167653) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_364(8167654) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_365(8167655) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_366(8167656) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_367(8167657) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_368(8167658) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_369(8167659) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_370(8167660) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_371(8167661) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_372(8167662) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_373(8167663) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_374(8167664) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_375(8167665) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_376(8167666) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_377(8167667) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_378(8167668) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_379(8167669) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_380(8167670) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_381(8167671) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_382(8167672) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_383(8167673) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_384(8167674) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_385(8167675) NodeList=r805u11n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_386(8167676) NodeList=r805u11n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_387(8167677) NodeList=r805u11n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_388(8167678) NodeList=r805u11n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_389(8167679) NodeList=r805u11n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_390(8167680) NodeList=r805u11n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_391(8167681) NodeList=r805u11n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_392(8167682) NodeList=r805u11n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_393(8167683) NodeList=r805u11n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_394(8167684) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_395(8167685) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_396(8167686) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_397(8167687) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_398(8167688) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_399(8167689) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_400(8167690) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_401(8167691) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_402(8167692) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_403(8167693) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_404(8167694) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_405(8167695) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_406(8167696) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_407(8167697) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_408(8167698) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_409(8167699) NodeList=r805u23n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_410(8167700) NodeList=r805u23n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_411(8167701) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_412(8167702) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_413(8167703) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_414(8167704) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_415(8167705) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_416(8167706) NodeList=r805u23n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_417(8167707) NodeList=r805u23n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_418(8167708) NodeList=r805u23n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_419(8167709) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_420(8167710) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_421(8167711) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_422(8167712) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_423(8167713) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_424(8167714) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_425(8167715) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_426(8167716) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_427(8167717) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_428(8167718) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_429(8167719) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_430(8167720) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_431(8167721) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_432(8167722) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_433(8167723) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_434(8167724) NodeList=r805u25n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_435(8167725) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_436(8167726) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_437(8167727) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_438(8167728) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_439(8167729) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_440(8167730) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_441(8167731) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_442(8167732) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_443(8167733) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_444(8167734) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_445(8167735) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_446(8167736) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_447(8167737) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_448(8167738) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_449(8167739) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_450(8167740) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_451(8167741) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_452(8167742) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_453(8167743) NodeList=r805u27n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_454(8167744) NodeList=r805u27n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_455(8167745) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_456(8167746) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_457(8167747) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_458(8167748) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_459(8167749) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_460(8167750) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_461(8167751) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_462(8167752) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_463(8167753) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_464(8167754) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_465(8167755) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_466(8167756) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_467(8167757) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_468(8167758) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_469(8167759) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_470(8167760) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_471(8167761) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_472(8167762) NodeList=r912u32n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_473(8167763) NodeList=r912u32n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_474(8167764) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_475(8167765) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_476(8167766) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_477(8167767) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_478(8167768) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_479(8167769) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_480(8167770) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_481(8167771) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_482(8167772) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_483(8167773) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_484(8167774) NodeList=r912u36n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_485(8167775) NodeList=r912u36n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_486(8167776) NodeList=r912u36n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_487(8167777) NodeList=r912u36n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_488(8167778) NodeList=r912u36n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_489(8167779) NodeList=r912u36n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_490(8167780) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_491(8167781) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_492(8167782) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_493(8167783) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_494(8167784) NodeList=r912u36n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_495(8167785) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_496(8167786) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_497(8167787) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_498(8167788) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_499(8167789) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_500(8167790) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_501(8167791) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_502(8167792) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_503(8167793) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_504(8167794) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_505(8167795) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_506(8167796) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_507(8167797) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_508(8167798) NodeList=r913u01n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_509(8167799) NodeList=r913u01n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_510(8167800) NodeList=r913u05n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_511(8167801) NodeList=r913u05n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_512(8167802) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_513(8167803) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_514(8167804) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_515(8167805) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_516(8167806) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_517(8167807) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_518(8167808) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_519(8167809) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_520(8167810) NodeList=r913u05n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_521(8167811) NodeList=r913u05n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_522(8167812) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_523(8167813) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_524(8167814) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_525(8167815) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_526(8167816) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_527(8167817) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_528(8167818) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_529(8167819) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_530(8167820) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_531(8167821) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_532(8167822) NodeList=r913u09n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_533(8167823) NodeList=r913u09n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_534(8167824) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_535(8167825) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_536(8167826) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_537(8167827) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_538(8167828) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_539(8167829) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_540(8167830) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_541(8167831) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_542(8167832) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_543(8167833) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_544(8167834) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_545(8167835) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_546(8167836) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_547(8167837) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_548(8167838) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_549(8167839) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_550(8167840) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_551(8167841) NodeList=r913u13n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_552(8167842) NodeList=r913u13n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_553(8167843) NodeList=r913u13n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_554(8167844) NodeList=r913u13n04 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_555(8167845) NodeList=r913u24n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_556(8167846) NodeList=r913u24n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_557(8167847) NodeList=r913u24n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_558(8167848) NodeList=r913u24n01 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_559(8167849) NodeList=r913u24n03 #CPUs=1 Partition=day Nov 05 09:25:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_560(8167850) NodeList=r913u24n03 #CPUs=1 Partition=day Nov 05 09:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6321(8163751) WEXITSTATUS 1 Nov 05 09:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6321(8163751) done Nov 05 09:25:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1992(8164130) WEXITSTATUS 0 Nov 05 09:25:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1992(8164130) done Nov 05 09:25:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:25:57 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_839(7573008) Nov 05 09:25:58 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:25:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1923(8163279) WEXITSTATUS 0 Nov 05 09:26:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1923(8163279) done Nov 05 09:26:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7098(8165654) WEXITSTATUS 1 Nov 05 09:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7098(8165654) done Nov 05 09:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7388(8166140) WEXITSTATUS 1 Nov 05 09:26:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7388(8166140) done Nov 05 09:26:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_524(8167814) WEXITSTATUS 0 Nov 05 09:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_524(8167814) done Nov 05 09:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_526(8167816) WEXITSTATUS 0 Nov 05 09:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_526(8167816) done Nov 05 09:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_525(8167815) WEXITSTATUS 0 Nov 05 09:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_525(8167815) done Nov 05 09:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_522(8167812) WEXITSTATUS 0 Nov 05 09:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_522(8167812) done Nov 05 09:26:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_523(8167813) WEXITSTATUS 0 Nov 05 09:26:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_523(8167813) done Nov 05 09:26:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_370(8167660) WEXITSTATUS 0 Nov 05 09:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_370(8167660) done Nov 05 09:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_372(8167662) WEXITSTATUS 0 Nov 05 09:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_372(8167662) done Nov 05 09:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_373(8167663) WEXITSTATUS 0 Nov 05 09:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_373(8167663) done Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_375(8167665) WEXITSTATUS 0 Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_375(8167665) done Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_376(8167666) WEXITSTATUS 0 Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_376(8167666) done Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_379(8167669) WEXITSTATUS 0 Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_379(8167669) done Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_378(8167668) WEXITSTATUS 0 Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_378(8167668) done Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_377(8167667) WEXITSTATUS 0 Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_377(8167667) done Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_388(8167678) WEXITSTATUS 0 Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_388(8167678) done Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_387(8167677) WEXITSTATUS 0 Nov 05 09:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_387(8167677) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6824(8165003) WEXITSTATUS 1 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6824(8165003) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_374(8167664) WEXITSTATUS 0 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_374(8167664) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_416(8167706) WEXITSTATUS 0 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_416(8167706) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_417(8167707) WEXITSTATUS 0 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_417(8167707) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_418(8167708) WEXITSTATUS 0 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_418(8167708) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_532(8167822) WEXITSTATUS 0 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_532(8167822) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_520(8167810) WEXITSTATUS 0 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_520(8167810) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_521(8167811) WEXITSTATUS 0 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_521(8167811) done Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_533(8167823) WEXITSTATUS 0 Nov 05 09:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_533(8167823) done Nov 05 09:26:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_492(8167782) WEXITSTATUS 0 Nov 05 09:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_492(8167782) done Nov 05 09:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_425(8167715) WEXITSTATUS 0 Nov 05 09:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_425(8167715) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_490(8167780) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_490(8167780) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_550(8167840) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_550(8167840) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_454(8167744) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_454(8167744) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_453(8167743) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_453(8167743) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_472(8167762) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_472(8167762) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_473(8167763) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_473(8167763) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_381(8167671) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_381(8167671) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_384(8167674) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_384(8167674) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_369(8167659) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_369(8167659) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_367(8167657) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_367(8167657) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_368(8167658) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_368(8167658) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_366(8167656) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_366(8167656) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_427(8167717) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_427(8167717) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_508(8167798) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_508(8167798) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_382(8167672) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_382(8167672) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_359(8167649) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_359(8167649) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_360(8167650) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_360(8167650) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_484(8167774) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_484(8167774) done Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_485(8167775) WEXITSTATUS 0 Nov 05 09:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_485(8167775) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_398(8167688) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_398(8167688) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_396(8167686) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_396(8167686) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_557(8167847) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_557(8167847) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_556(8167846) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_556(8167846) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_493(8167783) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_493(8167783) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_397(8167687) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_397(8167687) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_491(8167781) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_491(8167781) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_509(8167799) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_509(8167799) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_380(8167670) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_380(8167670) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_528(8167818) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_528(8167818) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_383(8167673) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_383(8167673) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_433(8167723) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_433(8167723) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_531(8167821) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_531(8167821) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_530(8167820) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_530(8167820) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_414(8167704) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_414(8167704) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_431(8167721) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_431(8167721) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_371(8167661) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_371(8167661) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_395(8167685) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_395(8167685) done Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_394(8167684) WEXITSTATUS 0 Nov 05 09:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_394(8167684) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_465(8167755) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_465(8167755) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_413(8167703) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_413(8167703) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_424(8167714) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_424(8167714) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_548(8167838) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_548(8167838) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_471(8167761) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_471(8167761) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_469(8167759) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_469(8167759) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_467(8167757) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_467(8167757) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_386(8167676) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_386(8167676) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_462(8167752) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_462(8167752) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_422(8167712) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_422(8167712) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_411(8167701) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_411(8167701) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_412(8167702) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_412(8167702) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_415(8167705) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_415(8167705) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_385(8167675) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_385(8167675) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_434(8167724) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_434(8167724) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_442(8167732) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_442(8167732) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_443(8167733) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_443(8167733) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_428(8167718) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_428(8167718) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_426(8167716) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_426(8167716) done Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_440(8167730) WEXITSTATUS 0 Nov 05 09:26:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_440(8167730) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_441(8167731) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_441(8167731) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_444(8167734) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_444(8167734) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_529(8167819) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_529(8167819) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_451(8167741) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_451(8167741) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_547(8167837) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_547(8167837) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_447(8167737) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_447(8167737) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_406(8167696) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_406(8167696) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_478(8167768) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_478(8167768) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_549(8167839) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_549(8167839) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_527(8167817) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_527(8167817) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_402(8167692) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_402(8167692) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_476(8167766) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_476(8167766) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_474(8167764) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_474(8167764) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_545(8167835) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_545(8167835) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_475(8167765) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_475(8167765) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_477(8167767) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_477(8167767) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_544(8167834) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_544(8167834) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_408(8167698) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_408(8167698) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_448(8167738) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_448(8167738) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_538(8167828) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_538(8167828) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_403(8167693) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_403(8167693) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_400(8167690) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_400(8167690) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_450(8167740) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_450(8167740) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_543(8167833) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_543(8167833) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_546(8167836) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_546(8167836) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_421(8167711) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_421(8167711) done Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_539(8167829) WEXITSTATUS 0 Nov 05 09:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_539(8167829) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_405(8167695) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_405(8167695) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_407(8167697) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_407(8167697) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_404(8167694) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_404(8167694) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_470(8167760) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_470(8167760) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_401(8167691) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_401(8167691) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_399(8167689) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_399(8167689) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_468(8167758) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_468(8167758) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_537(8167827) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_537(8167827) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_540(8167830) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_540(8167830) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_423(8167713) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_423(8167713) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_445(8167735) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_445(8167735) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_452(8167742) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_452(8167742) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_419(8167709) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_419(8167709) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_420(8167710) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_420(8167710) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_536(8167826) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_536(8167826) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_446(8167736) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_446(8167736) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_449(8167739) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_449(8167739) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_534(8167824) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_534(8167824) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_466(8167756) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_466(8167756) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_464(8167754) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_464(8167754) done Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_463(8167753) WEXITSTATUS 0 Nov 05 09:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_463(8167753) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_541(8167831) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_541(8167831) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_535(8167825) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_535(8167825) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_542(8167832) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_542(8167832) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_429(8167719) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_429(8167719) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_363(8167653) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_363(8167653) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_430(8167720) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_430(8167720) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_432(8167722) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_432(8167722) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_364(8167654) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_364(8167654) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_361(8167651) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_361(8167651) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_365(8167655) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_365(8167655) done Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_362(8167652) WEXITSTATUS 0 Nov 05 09:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_362(8167652) done Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_439(8167729) WEXITSTATUS 0 Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_439(8167729) done Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_436(8167726) WEXITSTATUS 0 Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_436(8167726) done Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_437(8167727) WEXITSTATUS 0 Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_437(8167727) done Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_435(8167725) WEXITSTATUS 0 Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_435(8167725) done Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_438(8167728) WEXITSTATUS 0 Nov 05 09:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_438(8167728) done Nov 05 09:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3907(8166737) WEXITSTATUS 0 Nov 05 09:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3907(8166737) done Nov 05 09:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_459(8167749) WEXITSTATUS 0 Nov 05 09:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_459(8167749) done Nov 05 09:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_461(8167751) WEXITSTATUS 0 Nov 05 09:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_461(8167751) done Nov 05 09:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_455(8167745) WEXITSTATUS 0 Nov 05 09:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_455(8167745) done Nov 05 09:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_458(8167748) WEXITSTATUS 0 Nov 05 09:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_458(8167748) done Nov 05 09:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_457(8167747) WEXITSTATUS 0 Nov 05 09:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_457(8167747) done Nov 05 09:26:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8167851 InitPrio=57465 usec=4042 Nov 05 09:26:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167851 NodeList=r801u30n01 #CPUs=2 Partition=pi_lora Nov 05 09:26:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_460(8167750) WEXITSTATUS 0 Nov 05 09:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_460(8167750) done Nov 05 09:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_456(8167746) WEXITSTATUS 0 Nov 05 09:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_456(8167746) done Nov 05 09:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_551(8167841) WEXITSTATUS 0 Nov 05 09:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_551(8167841) done Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1965(8163693) WEXITSTATUS 0 Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1965(8163693) done Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_503(8167793) WEXITSTATUS 0 Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_503(8167793) done Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_504(8167794) WEXITSTATUS 0 Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_504(8167794) done Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_505(8167795) WEXITSTATUS 0 Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_505(8167795) done Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_507(8167797) WEXITSTATUS 0 Nov 05 09:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_507(8167797) done Nov 05 09:26:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_506(8167796) WEXITSTATUS 0 Nov 05 09:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_506(8167796) done Nov 05 09:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_511(8167801) WEXITSTATUS 0 Nov 05 09:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_511(8167801) done Nov 05 09:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_510(8167800) WEXITSTATUS 0 Nov 05 09:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_510(8167800) done Nov 05 09:26:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_560(8167850) WEXITSTATUS 0 Nov 05 09:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_560(8167850) done Nov 05 09:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_559(8167849) WEXITSTATUS 0 Nov 05 09:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_559(8167849) done Nov 05 09:26:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_489(8167779) WEXITSTATUS 0 Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_489(8167779) done Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_486(8167776) WEXITSTATUS 0 Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_486(8167776) done Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_488(8167778) WEXITSTATUS 0 Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_488(8167778) done Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_500(8167790) WEXITSTATUS 0 Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_500(8167790) done Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_497(8167787) WEXITSTATUS 0 Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_497(8167787) done Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_496(8167786) WEXITSTATUS 0 Nov 05 09:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_496(8167786) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_518(8167808) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_518(8167808) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_516(8167806) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_516(8167806) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_519(8167809) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_519(8167809) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_512(8167802) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_512(8167802) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_514(8167804) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_514(8167804) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_515(8167805) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_515(8167805) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_513(8167803) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_513(8167803) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_517(8167807) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_517(8167807) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_502(8167792) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_502(8167792) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_498(8167788) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_498(8167788) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_501(8167791) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_501(8167791) done Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_499(8167789) WEXITSTATUS 0 Nov 05 09:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_499(8167789) done Nov 05 09:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_487(8167777) WEXITSTATUS 0 Nov 05 09:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_487(8167777) done Nov 05 09:26:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_483(8167773) WEXITSTATUS 0 Nov 05 09:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_483(8167773) done Nov 05 09:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_479(8167769) WEXITSTATUS 0 Nov 05 09:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_479(8167769) done Nov 05 09:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_482(8167772) WEXITSTATUS 0 Nov 05 09:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_482(8167772) done Nov 05 09:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_481(8167771) WEXITSTATUS 0 Nov 05 09:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_481(8167771) done Nov 05 09:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_480(8167770) WEXITSTATUS 0 Nov 05 09:26:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_480(8167770) done Nov 05 09:26:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8167852 NodeList=(null) usec=26639 Nov 05 09:26:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167852 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:26:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3915(8167077) WEXITSTATUS 0 Nov 05 09:26:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3915(8167077) done Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_561(8167853) NodeList=r807u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_562(8167854) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_563(8167855) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_564(8167856) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_565(8167857) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_566(8167858) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_567(8167859) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_568(8167860) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_569(8167861) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_570(8167862) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_571(8167863) NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_572(8167864) NodeList=r909u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_573(8167865) NodeList=r909u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_574(8167866) NodeList=r909u10n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_575(8167867) NodeList=r909u10n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_576(8167868) NodeList=r909u10n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_577(8167869) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_578(8167870) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_579(8167871) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_580(8167872) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_581(8167873) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_582(8167874) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_583(8167875) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_584(8167876) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_585(8167877) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_586(8167878) NodeList=r805u06n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_587(8167879) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_588(8167880) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_589(8167881) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_590(8167882) NodeList=r805u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_591(8167883) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_592(8167884) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_593(8167885) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_594(8167886) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_595(8167887) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_596(8167888) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_597(8167889) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_598(8167890) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_599(8167891) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_600(8167892) NodeList=r805u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_601(8167893) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_602(8167894) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_603(8167895) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_604(8167896) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_605(8167897) NodeList=r805u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_606(8167898) NodeList=r805u11n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_607(8167899) NodeList=r805u11n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_608(8167900) NodeList=r805u11n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_609(8167901) NodeList=r805u11n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_610(8167902) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_611(8167903) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_612(8167904) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_613(8167905) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_614(8167906) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_615(8167907) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_616(8167908) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_617(8167909) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_618(8167910) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_619(8167911) NodeList=r805u14n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_620(8167912) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_621(8167913) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_622(8167914) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_623(8167915) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_624(8167916) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_625(8167917) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_626(8167918) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_627(8167919) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_628(8167920) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_629(8167921) NodeList=r805u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_630(8167922) NodeList=r805u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_631(8167923) NodeList=r805u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_632(8167924) NodeList=r805u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_633(8167925) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_634(8167926) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_635(8167927) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_636(8167928) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_637(8167929) NodeList=r805u23n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_638(8167930) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_639(8167931) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_640(8167932) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_641(8167933) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_642(8167934) NodeList=r805u25n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_643(8167935) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_644(8167936) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_645(8167937) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_646(8167938) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_647(8167939) NodeList=r805u25n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_648(8167940) NodeList=r805u25n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_649(8167941) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_650(8167942) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_651(8167943) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_652(8167944) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_653(8167945) NodeList=r805u25n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_654(8167946) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_655(8167947) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_656(8167948) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_657(8167949) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_658(8167950) NodeList=r805u27n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_659(8167951) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_660(8167952) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_661(8167953) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_662(8167954) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_663(8167955) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_664(8167956) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_665(8167957) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_666(8167958) NodeList=r805u27n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_667(8167959) NodeList=r805u27n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_668(8167960) NodeList=r805u27n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_669(8167961) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_670(8167962) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_671(8167963) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_672(8167964) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_673(8167965) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_674(8167966) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_675(8167967) NodeList=r912u28n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_676(8167968) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_677(8167969) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_678(8167970) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_679(8167971) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_680(8167972) NodeList=r912u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_681(8167973) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_682(8167974) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_683(8167975) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_684(8167976) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_685(8167977) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_686(8167978) NodeList=r912u32n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_687(8167979) NodeList=r912u32n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_688(8167980) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_689(8167981) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_690(8167982) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_691(8167983) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_692(8167984) NodeList=r912u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_693(8167985) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_694(8167986) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_695(8167987) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_696(8167988) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_697(8167989) NodeList=r912u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_698(8167990) NodeList=r912u36n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_699(8167991) NodeList=r912u36n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_700(8167992) NodeList=r912u36n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_701(8167993) NodeList=r912u36n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_702(8167994) NodeList=r912u36n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_703(8167995) NodeList=r912u36n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_704(8167996) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_705(8167997) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_706(8167998) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_707(8167999) NodeList=r912u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_708(8168000) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_709(8168001) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_710(8168002) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_711(8168003) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_712(8168004) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_713(8168005) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_714(8168006) NodeList=r913u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_715(8168007) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_716(8168008) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_717(8168009) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_718(8168010) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_719(8168011) NodeList=r913u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_720(8168012) NodeList=r913u01n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_721(8168013) NodeList=r913u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_722(8168014) NodeList=r913u05n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_723(8168015) NodeList=r913u05n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_724(8168016) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_725(8168017) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_726(8168018) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_727(8168019) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_728(8168020) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_729(8168021) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_730(8168022) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_731(8168023) NodeList=r913u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_732(8168024) NodeList=r913u05n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_733(8168025) NodeList=r913u05n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_734(8168026) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_735(8168027) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_736(8168028) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_737(8168029) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_738(8168030) NodeList=r913u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_739(8168031) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_740(8168032) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_741(8168033) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_742(8168034) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_743(8168035) NodeList=r913u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_744(8168036) NodeList=r913u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_745(8168037) NodeList=r913u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_746(8168038) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_747(8168039) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_748(8168040) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_749(8168041) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_750(8168042) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_751(8168043) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_752(8168044) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_753(8168045) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_754(8168046) NodeList=r913u13n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_755(8168047) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_756(8168048) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_757(8168049) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_758(8168050) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_759(8168051) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_760(8168052) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_761(8168053) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_762(8168054) NodeList=r913u13n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_763(8168055) NodeList=r913u13n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_764(8168056) NodeList=r913u24n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_765(8168057) NodeList=r913u24n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_766(8168058) NodeList=r913u24n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_767(8168059) NodeList=r913u24n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_768(8168060) NodeList=r913u24n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_769(8168061) NodeList=r913u24n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_770(8168062) NodeList=r913u24n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_771(8168063) NodeList=r913u24n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_772(8168064) NodeList=r913u24n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_773(8168065) NodeList=r913u24n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_774(8168066) NodeList=r913u24n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_775(8168067) NodeList=r913u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_776(8168068) NodeList=r913u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_777(8168069) NodeList=r913u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_778(8168070) NodeList=r913u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_779(8168071) NodeList=r913u28n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_780(8168072) NodeList=r913u28n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_781(8168073) NodeList=r913u28n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_782(8168074) NodeList=r913u28n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_783(8168075) NodeList=r913u28n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_784(8168076) NodeList=r913u28n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_785(8168077) NodeList=r913u28n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_786(8168078) NodeList=r913u28n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_787(8168079) NodeList=r913u28n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_788(8168080) NodeList=r913u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_789(8168081) NodeList=r913u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_790(8168082) NodeList=r913u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_791(8168083) NodeList=r913u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_792(8168084) NodeList=r913u28n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_793(8168085) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_794(8168086) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_795(8168087) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_796(8168088) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_797(8168089) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_798(8168090) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_799(8168091) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_800(8168092) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_801(8168093) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_802(8168094) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_803(8168095) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_804(8168096) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_805(8168097) NodeList=r918u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_806(8168098) NodeList=r918u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_807(8168099) NodeList=r918u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_808(8168100) NodeList=r918u05n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_809(8168101) NodeList=r918u05n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_810(8168102) NodeList=r918u05n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_811(8168103) NodeList=r918u05n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_812(8168104) NodeList=r918u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_813(8168105) NodeList=r918u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_814(8168106) NodeList=r918u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_815(8168107) NodeList=r918u05n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_816(8168108) NodeList=r918u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_817(8168109) NodeList=r918u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_818(8168110) NodeList=r918u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_819(8168111) NodeList=r918u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_820(8168112) NodeList=r918u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_821(8168113) NodeList=r918u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_822(8168114) NodeList=r918u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_823(8168115) NodeList=r918u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_824(8168116) NodeList=r918u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_825(8168117) NodeList=r918u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_826(8168118) NodeList=r918u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_827(8168119) NodeList=r918u32n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_828(8168120) NodeList=r918u32n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_829(8168121) NodeList=r918u32n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_830(8168122) NodeList=r918u36n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_831(8168123) NodeList=r918u36n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_832(8168124) NodeList=r918u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_833(8168125) NodeList=r918u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_834(8168126) NodeList=r918u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_835(8168127) NodeList=r918u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_836(8168128) NodeList=r918u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_837(8168129) NodeList=r918u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_838(8168130) NodeList=r919u01n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_839(8168131) NodeList=r919u01n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_840(8168132) NodeList=r919u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_841(8168133) NodeList=r919u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_842(8168134) NodeList=r919u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_843(8168135) NodeList=r919u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_844(8168136) NodeList=r919u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_845(8168137) NodeList=r919u01n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_846(8168138) NodeList=r919u05n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_847(8168139) NodeList=r919u05n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_848(8168140) NodeList=r919u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_849(8168141) NodeList=r919u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_850(8168142) NodeList=r919u05n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_851(8168143) NodeList=r919u05n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_852(8168144) NodeList=r919u05n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_853(8168145) NodeList=r919u05n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_854(8168146) NodeList=r919u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_855(8168147) NodeList=r919u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_856(8168148) NodeList=r919u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_857(8168149) NodeList=r919u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_858(8168150) NodeList=r919u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_859(8168151) NodeList=r919u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_860(8168152) NodeList=r919u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_861(8168153) NodeList=r919u32n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_862(8168154) NodeList=r919u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_863(8168155) NodeList=r919u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_864(8168156) NodeList=r919u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_865(8168157) NodeList=r919u32n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_866(8168158) NodeList=r919u32n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_867(8168159) NodeList=r919u36n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_868(8168160) NodeList=r919u36n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_869(8168161) NodeList=r919u36n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_870(8168162) NodeList=r919u36n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_871(8168163) NodeList=r919u36n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_872(8168164) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_873(8168165) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_874(8168166) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_875(8168167) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_876(8168168) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_877(8168169) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_878(8168170) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_879(8168171) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_880(8168172) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_881(8168173) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_882(8168174) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_883(8168175) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_884(8168176) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_885(8168177) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_886(8168178) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_887(8168179) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_888(8168180) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_889(8168181) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_890(8168182) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_891(8168183) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_892(8168184) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_893(8168185) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_894(8168186) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_895(8168187) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_896(8168188) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_897(8168189) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_898(8168190) NodeList=r815u23n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_899(8168191) NodeList=r815u23n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_900(8168192) NodeList=r815u23n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_901(8168193) NodeList=r815u23n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_902(8168194) NodeList=r815u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_903(8168195) NodeList=r815u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_904(8168196) NodeList=r815u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_905(8168197) NodeList=r815u23n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_906(8168198) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_907(8168199) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_908(8168200) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_909(8168201) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_910(8168202) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_911(8168203) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_912(8168204) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_913(8168205) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_914(8168206) NodeList=r815u23n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_915(8168207) NodeList=r815u23n06 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_916(8168208) NodeList=r815u29n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_917(8168209) NodeList=r815u29n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_918(8168210) NodeList=r815u29n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_919(8168211) NodeList=r815u29n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_920(8168212) NodeList=r815u29n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_921(8168213) NodeList=r815u29n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_922(8168214) NodeList=r815u29n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_923(8168215) NodeList=r815u29n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_924(8168216) NodeList=r815u29n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_925(8168217) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_926(8168218) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_927(8168219) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_928(8168220) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_929(8168221) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_930(8168222) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_931(8168223) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_932(8168224) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_933(8168225) NodeList=r815u29n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_934(8168226) NodeList=r815u29n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_935(8168227) NodeList=r815u29n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_936(8168228) NodeList=r815u29n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_937(8168229) NodeList=r815u29n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_938(8168230) NodeList=r815u29n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_939(8168231) NodeList=r815u29n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_940(8168232) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_941(8168233) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_942(8168234) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_943(8168235) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_944(8168236) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_945(8168237) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_946(8168238) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_947(8168239) NodeList=r815u29n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_948(8168240) NodeList=r815u29n06 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_949(8168241) NodeList=r815u35n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_950(8168242) NodeList=r815u35n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_951(8168243) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_952(8168244) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_953(8168245) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_954(8168246) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_955(8168247) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_956(8168248) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_957(8168249) NodeList=r815u35n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_958(8168250) NodeList=r815u35n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_959(8168251) NodeList=r815u35n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_960(8168252) NodeList=r815u35n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_961(8168253) NodeList=r815u35n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_962(8168254) NodeList=r815u35n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_963(8168255) NodeList=r815u35n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_964(8168256) NodeList=r815u35n06 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_965(8168257) NodeList=r815u35n06 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_966(8168258) NodeList=r815u35n06 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_967(8168259) NodeList=r815u03n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_968(8168260) NodeList=r815u03n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_969(8168261) NodeList=r815u03n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_970(8168262) NodeList=r815u03n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_971(8168263) NodeList=r815u03n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_972(8168264) NodeList=r815u03n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_973(8168265) NodeList=r815u03n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_974(8168266) NodeList=r815u03n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_975(8168267) NodeList=r815u03n07 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_976(8168268) NodeList=r815u03n07 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_977(8168269) NodeList=r815u03n07 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_978(8168270) NodeList=r815u03n07 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_979(8168271) NodeList=r815u03n07 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_980(8168272) NodeList=r815u03n07 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_981(8168273) NodeList=r815u03n07 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_982(8168274) NodeList=r815u03n08 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_983(8168275) NodeList=r815u03n08 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_984(8168276) NodeList=r815u03n08 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_985(8168277) NodeList=r815u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_986(8168278) NodeList=r815u09n01 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_987(8168279) NodeList=r815u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_988(8168280) NodeList=r815u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_989(8168281) NodeList=r815u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_990(8168282) NodeList=r815u09n02 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_991(8168283) NodeList=r815u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_992(8168284) NodeList=r815u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_993(8168285) NodeList=r815u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_994(8168286) NodeList=r815u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_995(8168287) NodeList=r815u09n03 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_996(8168288) NodeList=r815u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_997(8168289) NodeList=r815u09n04 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_998(8168290) NodeList=r815u09n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167267_999(8167267) NodeList=r815u09n05 #CPUs=1 Partition=day Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167599 NodeList=r816u03n09 #CPUs=12 Partition=scavenge Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167600 NodeList=r816u03n12 #CPUs=12 Partition=scavenge Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167601 NodeList=r816u03n12 #CPUs=12 Partition=scavenge Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167602 NodeList=r816u09n09 #CPUs=12 Partition=scavenge Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167603 NodeList=r816u23n09 #CPUs=12 Partition=scavenge Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167604 NodeList=r816u23n10 #CPUs=12 Partition=scavenge Nov 05 09:26:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8167605 NodeList=r816u29n09 #CPUs=12 Partition=scavenge Nov 05 09:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_260(7571225) WEXITSTATUS 0 Nov 05 09:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_260(7571225) done Nov 05 09:26:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_5(8158165) WEXITSTATUS 1 Nov 05 09:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_5(8158165) done Nov 05 09:26:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7399(8166155) WEXITSTATUS 1 Nov 05 09:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7399(8166155) done Nov 05 09:26:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168291 InitPrio=10821 usec=3706 Nov 05 09:27:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168292 InitPrio=10821 usec=3904 Nov 05 09:27:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7623(8167068) WEXITSTATUS 1 Nov 05 09:27:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7623(8167068) done Nov 05 09:27:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168293 InitPrio=53313 usec=2860 Nov 05 09:27:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168294 InitPrio=10821 usec=2622 Nov 05 09:27:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168293 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:27:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_701(8167993) WEXITSTATUS 0 Nov 05 09:27:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_701(8167993) done Nov 05 09:27:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_948(8168240) WEXITSTATUS 0 Nov 05 09:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_948(8168240) done Nov 05 09:27:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168295 InitPrio=10821 usec=3607 Nov 05 09:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_986(8168278) WEXITSTATUS 0 Nov 05 09:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_986(8168278) done Nov 05 09:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_703(8167995) WEXITSTATUS 0 Nov 05 09:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_703(8167995) done Nov 05 09:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_985(8168277) WEXITSTATUS 0 Nov 05 09:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_985(8168277) done Nov 05 09:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_871(8168163) WEXITSTATUS 0 Nov 05 09:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_871(8168163) done Nov 05 09:27:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_702(8167994) WEXITSTATUS 0 Nov 05 09:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_702(8167994) done Nov 05 09:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_744(8168036) WEXITSTATUS 0 Nov 05 09:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_744(8168036) done Nov 05 09:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_745(8168037) WEXITSTATUS 0 Nov 05 09:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_745(8168037) done Nov 05 09:27:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_720(8168012) WEXITSTATUS 0 Nov 05 09:27:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_720(8168012) done Nov 05 09:27:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_700(8167992) WEXITSTATUS 0 Nov 05 09:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_700(8167992) done Nov 05 09:27:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting time_limit to 10080 for JobId=8153362 Nov 05 09:27:24 r912u23n01 slurmctld[163760]: slurmctld: _update_job: updating accounting Nov 05 09:27:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8153362 uid=11917 usec=12285 Nov 05 09:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_668(8167960) WEXITSTATUS 0 Nov 05 09:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_668(8167960) done Nov 05 09:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_581(8167873) WEXITSTATUS 0 Nov 05 09:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_581(8167873) done Nov 05 09:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_854(8168146) WEXITSTATUS 0 Nov 05 09:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_854(8168146) done Nov 05 09:27:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_580(8167872) WEXITSTATUS 0 Nov 05 09:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_580(8167872) done Nov 05 09:27:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168291 in pi_ohern on r801u32n02 Nov 05 09:27:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168292 in pi_ohern on r801u32n02 Nov 05 09:27:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168294 in pi_ohern on r917u13n01 Nov 05 09:27:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168295 in pi_ohern on r917u13n01 Nov 05 09:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3924(8168296) in gpu on r903u34n01 Nov 05 09:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3925(8168297) in gpu on r808u14n01 Nov 05 09:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3926(8168298) in gpu on r4513u29n01 Nov 05 09:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2820(8168299) in day on r805u27n03 Nov 05 09:27:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2821(8168300) in day on r815u03n07 Nov 05 09:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1983(8163993) WEXITSTATUS 0 Nov 05 09:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1983(8163993) done Nov 05 09:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_816(8168108) WEXITSTATUS 0 Nov 05 09:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_816(8168108) done Nov 05 09:27:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8167606 in scavenge on r815u03n12 Nov 05 09:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_826(8168118) WEXITSTATUS 0 Nov 05 09:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_826(8168118) done Nov 05 09:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_825(8168117) WEXITSTATUS 0 Nov 05 09:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_825(8168117) done Nov 05 09:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_607(8167899) WEXITSTATUS 0 Nov 05 09:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_607(8167899) done Nov 05 09:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_721(8168013) WEXITSTATUS 0 Nov 05 09:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_721(8168013) done Nov 05 09:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_838(8168130) WEXITSTATUS 0 Nov 05 09:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_838(8168130) done Nov 05 09:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_914(8168206) WEXITSTATUS 0 Nov 05 09:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_914(8168206) done Nov 05 09:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_743(8168035) WEXITSTATUS 0 Nov 05 09:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_743(8168035) done Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7665(8168301) in scavenge on r802u11n02 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7666(8168302) in scavenge on r802u30n03 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7667(8168303) in scavenge on r815u23n08 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7668(8168304) in scavenge on r806u09n04 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7669(8168305) in scavenge on r919u36n01 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7670(8168306) in scavenge on r815u09n07 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7671(8168307) in scavenge on r816u03n01 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7672(8168308) in scavenge on r816u03n03 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7673(8168309) in scavenge on r816u03n03 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7674(8168310) in scavenge on r816u03n05 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7675(8168311) in scavenge on r816u03n08 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7676(8168312) in scavenge on r816u09n05 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7677(8168313) in scavenge on r816u09n05 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7678(8168314) in scavenge on r816u09n05 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_589(8167881) WEXITSTATUS 0 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_589(8167881) done Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_723(8168015) WEXITSTATUS 0 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_723(8168015) done Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_849(8168141) WEXITSTATUS 0 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_849(8168141) done Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_609(8167901) WEXITSTATUS 0 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_609(8167901) done Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_848(8168140) WEXITSTATUS 0 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_848(8168140) done Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_608(8167900) WEXITSTATUS 0 Nov 05 09:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_608(8167900) done Nov 05 09:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_722(8168014) WEXITSTATUS 0 Nov 05 09:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_722(8168014) done Nov 05 09:27:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1004(7573173) in scavenge on r801u30n01 Nov 05 09:27:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_437(8114704) in scavenge on r816u03n12 Nov 05 09:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2116(8164571) WEXITSTATUS 0 Nov 05 09:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2116(8164571) done Nov 05 09:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_648(8167940) WEXITSTATUS 0 Nov 05 09:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_648(8167940) done Nov 05 09:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_811(8168103) WEXITSTATUS 0 Nov 05 09:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_811(8168103) done Nov 05 09:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_847(8168139) WEXITSTATUS 0 Nov 05 09:27:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_847(8168139) done Nov 05 09:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_599(8167891) WEXITSTATUS 0 Nov 05 09:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_599(8167891) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_684(8167976) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_684(8167976) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_682(8167974) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_682(8167974) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_813(8168105) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_813(8168105) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_707(8167999) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_707(8167999) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_814(8168106) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_814(8168106) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_830(8168122) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_830(8168122) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_683(8167975) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_683(8167975) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_763(8168055) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_763(8168055) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_904(8168196) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_904(8168196) done Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_705(8167997) WEXITSTATUS 0 Nov 05 09:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_705(8167997) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_902(8168194) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_902(8168194) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_903(8168195) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_903(8168195) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_865(8168157) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_865(8168157) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_687(8167979) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_687(8167979) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_806(8168098) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_806(8168098) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_805(8168097) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_805(8168097) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_639(8167931) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_639(8167931) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_807(8168099) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_807(8168099) done Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_494(8167784) WEXITSTATUS 0 Nov 05 09:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_494(8167784) done Nov 05 09:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_840(8168132) WEXITSTATUS 0 Nov 05 09:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_840(8168132) done Nov 05 09:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_972(8168264) WEXITSTATUS 0 Nov 05 09:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_972(8168264) done Nov 05 09:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_970(8168262) WEXITSTATUS 0 Nov 05 09:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_970(8168262) done Nov 05 09:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_971(8168263) WEXITSTATUS 0 Nov 05 09:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_971(8168263) done Nov 05 09:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_804(8168096) WEXITSTATUS 0 Nov 05 09:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_804(8168096) done Nov 05 09:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_641(8167933) WEXITSTATUS 0 Nov 05 09:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_641(8167933) done Nov 05 09:27:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_867(8168159) WEXITSTATUS 0 Nov 05 09:27:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_867(8168159) done Nov 05 09:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_575(8167867) WEXITSTATUS 0 Nov 05 09:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_575(8167867) done Nov 05 09:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_820(8168112) WEXITSTATUS 0 Nov 05 09:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_820(8168112) done Nov 05 09:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_558(8167848) WEXITSTATUS 0 Nov 05 09:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_558(8167848) done Nov 05 09:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_574(8167866) WEXITSTATUS 0 Nov 05 09:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_574(8167866) done Nov 05 09:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_576(8167868) WEXITSTATUS 0 Nov 05 09:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_576(8167868) done Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_653(8167945) WEXITSTATUS 0 Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_653(8167945) done Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_927(8168219) WEXITSTATUS 0 Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_927(8168219) done Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_932(8168224) WEXITSTATUS 0 Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_932(8168224) done Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_774(8168066) WEXITSTATUS 0 Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_774(8168066) done Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_410(8167700) WEXITSTATUS 0 Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_410(8167700) done Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_409(8167699) WEXITSTATUS 0 Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_409(8167699) done Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_555(8167845) WEXITSTATUS 0 Nov 05 09:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_555(8167845) done Nov 05 09:27:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_591(8167883) WEXITSTATUS 0 Nov 05 09:27:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_591(8167883) done Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_900(8168192) WEXITSTATUS 0 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_900(8168192) done Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_620(8167912) WEXITSTATUS 0 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_620(8167912) done Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_624(8167916) WEXITSTATUS 0 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_624(8167916) done Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_587(8167879) WEXITSTATUS 0 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_587(8167879) done Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_621(8167913) WEXITSTATUS 0 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_621(8167913) done Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_553(8167843) WEXITSTATUS 0 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_553(8167843) done Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_554(8167844) WEXITSTATUS 0 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_554(8167844) done Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_552(8167842) WEXITSTATUS 0 Nov 05 09:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_552(8167842) done Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_590(8167882) WEXITSTATUS 0 Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_590(8167882) done Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_588(8167880) WEXITSTATUS 0 Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_588(8167880) done Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_693(8167985) WEXITSTATUS 0 Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_693(8167985) done Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_990(8168282) WEXITSTATUS 0 Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_990(8168282) done Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_596(8167888) WEXITSTATUS 0 Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_596(8167888) done Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_695(8167987) WEXITSTATUS 0 Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_695(8167987) done Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_841(8168133) WEXITSTATUS 0 Nov 05 09:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_841(8168133) done Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2822(8168315) NodeList=r912u28n04 #CPUs=1 Partition=day Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_717(8168009) WEXITSTATUS 0 Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_717(8168009) done Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_626(8167918) WEXITSTATUS 0 Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_626(8167918) done Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_860(8168152) WEXITSTATUS 0 Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_860(8168152) done Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_845(8168137) WEXITSTATUS 0 Nov 05 09:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_845(8168137) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_747(8168039) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_747(8168039) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_358(8167648) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_358(8167648) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_751(8168043) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_751(8168043) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_842(8168134) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_842(8168134) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_843(8168135) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_843(8168135) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_616(8167908) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_616(8167908) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_692(8167984) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_692(8167984) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_600(8167892) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_600(8167892) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_598(8167890) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_598(8167890) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_615(8167907) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_615(8167907) done Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_595(8167887) WEXITSTATUS 0 Nov 05 09:27:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_595(8167887) done Nov 05 09:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_594(8167886) WEXITSTATUS 0 Nov 05 09:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_594(8167886) done Nov 05 09:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_857(8168149) WEXITSTATUS 0 Nov 05 09:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_857(8168149) done Nov 05 09:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_752(8168044) WEXITSTATUS 0 Nov 05 09:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_752(8168044) done Nov 05 09:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_915(8168207) WEXITSTATUS 0 Nov 05 09:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_915(8168207) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_776(8168068) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_776(8168068) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_989(8168281) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_989(8168281) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_778(8168070) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_778(8168070) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_765(8168057) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_765(8168057) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_995(8168287) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_995(8168287) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_992(8168284) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_992(8168284) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_393(8167683) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_393(8167683) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_628(8167920) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_628(8167920) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_856(8168148) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_856(8168148) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_391(8167681) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_391(8167681) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_630(8167922) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_630(8167922) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_988(8168280) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_988(8168280) done Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_987(8168279) WEXITSTATUS 0 Nov 05 09:27:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_987(8168279) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_822(8168114) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_822(8168114) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_821(8168113) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_821(8168113) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_823(8168115) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_823(8168115) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_777(8168069) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_777(8168069) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_764(8168056) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_764(8168056) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_625(8167917) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_625(8167917) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_779(8168071) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_779(8168071) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_775(8168067) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_775(8168067) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_631(8167923) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_631(8167923) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_392(8167682) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_392(8167682) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_389(8167679) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_389(8167679) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_390(8167680) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_390(8167680) done Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_629(8167921) WEXITSTATUS 0 Nov 05 09:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_629(8167921) done Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_627(8167919) WEXITSTATUS 0 Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_627(8167919) done Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_592(8167884) WEXITSTATUS 0 Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_592(8167884) done Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_593(8167885) WEXITSTATUS 0 Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_593(8167885) done Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_800(8168092) WEXITSTATUS 0 Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_800(8168092) done Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_802(8168094) WEXITSTATUS 0 Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_802(8168094) done Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_801(8168093) WEXITSTATUS 0 Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_801(8168093) done Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_799(8168091) WEXITSTATUS 0 Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_799(8168091) done Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_919(8168211) WEXITSTATUS 0 Nov 05 09:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_919(8168211) done Nov 05 09:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_831(8168123) WEXITSTATUS 0 Nov 05 09:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_831(8168123) done Nov 05 09:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_808(8168100) WEXITSTATUS 0 Nov 05 09:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_808(8168100) done Nov 05 09:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_836(8168128) WEXITSTATUS 0 Nov 05 09:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_836(8168128) done Nov 05 09:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_964(8168256) WEXITSTATUS 0 Nov 05 09:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_964(8168256) done Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_922(8168214) WEXITSTATUS 0 Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_922(8168214) done Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_579(8167871) WEXITSTATUS 0 Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_579(8167871) done Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_953(8168245) WEXITSTATUS 0 Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_953(8168245) done Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_654(8167946) WEXITSTATUS 0 Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_654(8167946) done Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_584(8167876) WEXITSTATUS 0 Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_584(8167876) done Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_735(8168027) WEXITSTATUS 0 Nov 05 09:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_735(8168027) done Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_732(8168024) WEXITSTATUS 0 Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_732(8168024) done Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_736(8168028) WEXITSTATUS 0 Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_736(8168028) done Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_734(8168026) WEXITSTATUS 0 Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_734(8168026) done Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_733(8168025) WEXITSTATUS 0 Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_733(8168025) done Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_792(8168084) WEXITSTATUS 0 Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_792(8168084) done Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_921(8168213) WEXITSTATUS 0 Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_921(8168213) done Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_920(8168212) WEXITSTATUS 0 Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_920(8168212) done Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_924(8168216) WEXITSTATUS 0 Nov 05 09:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_924(8168216) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_789(8168081) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_789(8168081) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_791(8168083) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_791(8168083) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_853(8168145) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_853(8168145) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_656(8167948) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_656(8167948) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_658(8167950) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_658(8167950) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_698(8167990) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_698(8167990) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_832(8168124) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_832(8168124) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_699(8167991) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_699(8167991) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_834(8168126) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_834(8168126) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_851(8168143) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_851(8168143) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_833(8168125) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_833(8168125) done Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_671(8167963) WEXITSTATUS 0 Nov 05 09:27:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_671(8167963) done Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3994(8146258) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_796(8168088) WEXITSTATUS 0 Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_796(8168088) done Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_674(8167966) WEXITSTATUS 0 Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_674(8167966) done Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_759(8168051) WEXITSTATUS 0 Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_759(8168051) done Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_601(8167893) WEXITSTATUS 0 Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_601(8167893) done Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_604(8167896) WEXITSTATUS 0 Nov 05 09:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_604(8167896) done Nov 05 09:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_785(8168077) WEXITSTATUS 0 Nov 05 09:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_785(8168077) done Nov 05 09:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_730(8168022) WEXITSTATUS 0 Nov 05 09:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_730(8168022) done Nov 05 09:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_963(8168255) WEXITSTATUS 0 Nov 05 09:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_963(8168255) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3994(8146258) Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_850(8168142) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_850(8168142) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_780(8168072) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_780(8168072) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_954(8168246) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_954(8168246) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_952(8168244) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_952(8168244) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_677(8167969) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_677(8167969) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_670(8167962) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_670(8167962) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_793(8168085) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_793(8168085) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_955(8168247) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_955(8168247) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_676(8167968) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_676(8167968) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_728(8168020) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_728(8168020) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_673(8167965) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_673(8167965) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_977(8168269) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_977(8168269) done Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_787(8168079) WEXITSTATUS 0 Nov 05 09:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_787(8168079) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_783(8168075) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_783(8168075) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_750(8168042) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_750(8168042) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_612(8167904) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_612(8167904) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_678(8167970) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_678(8167970) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_824(8168116) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_824(8168116) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_956(8168248) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_956(8168248) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_725(8168017) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_725(8168017) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_979(8168271) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_979(8168271) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_975(8168267) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_975(8168267) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_976(8168268) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_976(8168268) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_613(8167905) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_613(8167905) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_749(8168041) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_749(8168041) done Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_753(8168045) WEXITSTATUS 0 Nov 05 09:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_753(8168045) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_957(8168249) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_957(8168249) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_495(8167785) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_495(8167785) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_912(8168204) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_912(8168204) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_709(8168001) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_709(8168001) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_754(8168046) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_754(8168046) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_714(8168006) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_714(8168006) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_633(8167925) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_633(8167925) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_711(8168003) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_711(8168003) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_662(8167954) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_662(8167954) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7517(8166528) WEXITSTATUS 1 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7517(8166528) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_782(8168074) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_782(8168074) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_784(8168076) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_784(8168076) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_746(8168038) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_746(8168038) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_610(8167902) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_610(8167902) done Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_930(8168222) WEXITSTATUS 0 Nov 05 09:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_930(8168222) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_614(8167906) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_614(8167906) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_962(8168254) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_962(8168254) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_961(8168253) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_961(8168253) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_708(8168000) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_708(8168000) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_959(8168251) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_959(8168251) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_960(8168252) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_960(8168252) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_925(8168217) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_925(8168217) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_958(8168250) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_958(8168250) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_568(8167860) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_568(8167860) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_928(8168220) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_928(8168220) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_713(8168005) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_713(8168005) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_665(8167957) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_665(8167957) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_712(8168004) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_712(8168004) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_710(8168002) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_710(8168002) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_635(8167927) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_635(8167927) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_637(8167929) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_637(8167929) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_933(8168225) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_933(8168225) done Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_929(8168221) WEXITSTATUS 0 Nov 05 09:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_929(8168221) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_659(8167951) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_659(8167951) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_660(8167952) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_660(8167952) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_926(8168218) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_926(8168218) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_666(8167958) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_666(8167958) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_570(8167862) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_570(8167862) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_888(8168180) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_888(8168180) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_875(8168167) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_875(8168167) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_909(8168201) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_909(8168201) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6861(8165074) WEXITSTATUS 1 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6861(8165074) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_661(8167953) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_661(8167953) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_886(8168178) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_886(8168178) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_663(8167955) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_663(8167955) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_664(8167956) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_664(8167956) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_911(8168203) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_911(8168203) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_906(8168198) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_906(8168198) done Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_768(8168060) WEXITSTATUS 0 Nov 05 09:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_768(8168060) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_913(8168205) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_913(8168205) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_907(8168199) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_907(8168199) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_910(8168202) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_910(8168202) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_908(8168200) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_908(8168200) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_769(8168061) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_769(8168061) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_874(8168166) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_874(8168166) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_767(8168059) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_767(8168059) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_766(8168058) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_766(8168058) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_897(8168189) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_897(8168189) done Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_879(8168171) WEXITSTATUS 0 Nov 05 09:28:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_879(8168171) done Nov 05 09:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_937(8168229) WEXITSTATUS 0 Nov 05 09:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_937(8168229) done Nov 05 09:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_934(8168226) WEXITSTATUS 0 Nov 05 09:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_934(8168226) done Nov 05 09:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_935(8168227) WEXITSTATUS 0 Nov 05 09:28:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_935(8168227) done Nov 05 09:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_938(8168230) WEXITSTATUS 0 Nov 05 09:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_938(8168230) done Nov 05 09:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_936(8168228) WEXITSTATUS 0 Nov 05 09:28:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_936(8168228) done Nov 05 09:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_680(8167972) WEXITSTATUS 0 Nov 05 09:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_680(8167972) done Nov 05 09:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7625(8167070) WEXITSTATUS 1 Nov 05 09:28:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7625(8167070) done Nov 05 09:28:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_862(8168154) WEXITSTATUS 0 Nov 05 09:28:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_862(8168154) done Nov 05 09:28:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_864(8168156) WEXITSTATUS 0 Nov 05 09:28:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_864(8168156) done Nov 05 09:28:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_863(8168155) WEXITSTATUS 0 Nov 05 09:28:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_863(8168155) done Nov 05 09:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_22(8165378) WEXITSTATUS 0 Nov 05 09:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_22(8165378) done Nov 05 09:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4384(8155314) WEXITSTATUS 1 Nov 05 09:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4384(8155314) done Nov 05 09:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_815(8168107) WEXITSTATUS 0 Nov 05 09:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_815(8168107) done Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160343 WEXITSTATUS 0 Nov 05 09:28:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160343 done Nov 05 09:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_846(8168138) WEXITSTATUS 0 Nov 05 09:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_846(8168138) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_894(8168186) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_894(8168186) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_877(8168169) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_877(8168169) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_893(8168185) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_893(8168185) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_878(8168170) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_878(8168170) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_895(8168187) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_895(8168187) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_885(8168177) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_885(8168177) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_881(8168173) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_881(8168173) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_892(8168184) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_892(8168184) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_883(8168175) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_883(8168175) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_891(8168183) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_891(8168183) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_873(8168165) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_873(8168165) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_880(8168172) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_880(8168172) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_889(8168181) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_889(8168181) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_890(8168182) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_890(8168182) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_872(8168164) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_872(8168164) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_882(8168174) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_882(8168174) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_887(8168179) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_887(8168179) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_896(8168188) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_896(8168188) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_876(8168168) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_876(8168168) done Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_884(8168176) WEXITSTATUS 0 Nov 05 09:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_884(8168176) done Nov 05 09:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7350(8166059) WEXITSTATUS 1 Nov 05 09:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7350(8166059) done Nov 05 09:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_798(8168090) WEXITSTATUS 0 Nov 05 09:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_798(8168090) done Nov 05 09:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_795(8168087) WEXITSTATUS 0 Nov 05 09:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_795(8168087) done Nov 05 09:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_797(8168089) WEXITSTATUS 0 Nov 05 09:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_797(8168089) done Nov 05 09:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_794(8168086) WEXITSTATUS 0 Nov 05 09:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_794(8168086) done Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160535 WEXITSTATUS 0 Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160535 done Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_739(8168031) WEXITSTATUS 0 Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_739(8168031) done Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_740(8168032) WEXITSTATUS 0 Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_740(8168032) done Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_741(8168033) WEXITSTATUS 0 Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_741(8168033) done Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_742(8168034) WEXITSTATUS 0 Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_742(8168034) done Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_623(8167915) WEXITSTATUS 0 Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_623(8167915) done Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_622(8167914) WEXITSTATUS 0 Nov 05 09:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_622(8167914) done Nov 05 09:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_835(8168127) WEXITSTATUS 0 Nov 05 09:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_835(8168127) done Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_339(7572507) Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_627(7572796) Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_687(7572856) Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_813(7572982) Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_815(7572984) Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_854(7573023) Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_858(7573027) Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_889(7573058) Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_844(8168136) WEXITSTATUS 0 Nov 05 09:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_844(8168136) done Nov 05 09:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_817(8168109) WEXITSTATUS 0 Nov 05 09:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_817(8168109) done Nov 05 09:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_818(8168110) WEXITSTATUS 0 Nov 05 09:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_818(8168110) done Nov 05 09:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_819(8168111) WEXITSTATUS 0 Nov 05 09:28:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_819(8168111) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_718(8168010) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_718(8168010) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_715(8168007) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_715(8168007) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_716(8168008) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_716(8168008) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_719(8168011) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_719(8168011) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_809(8168101) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_809(8168101) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_810(8168102) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_810(8168102) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_685(8167977) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_685(8167977) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_827(8168119) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_827(8168119) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_829(8168121) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_829(8168121) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_828(8168120) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_828(8168120) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_681(8167973) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_681(8167973) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_704(8167996) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_704(8167996) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_706(8167998) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_706(8167998) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_667(8167959) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_667(8167959) done Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_561(8167853) WEXITSTATUS 0 Nov 05 09:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_561(8167853) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_657(8167949) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_657(8167949) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_655(8167947) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_655(8167947) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_905(8168197) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_905(8168197) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_999(8167267) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_999(8167267) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_998(8168290) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_998(8168290) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_839(8168131) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_839(8168131) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_672(8167964) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_672(8167964) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_669(8167961) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_669(8167961) done Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_675(8167967) WEXITSTATUS 0 Nov 05 09:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_675(8167967) done Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_837(8168129) WEXITSTATUS 0 Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_837(8168129) done Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_951(8168243) WEXITSTATUS 0 Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_951(8168243) done Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_923(8168215) WEXITSTATUS 0 Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_923(8168215) done Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_618(8167910) WEXITSTATUS 0 Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_618(8167910) done Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_617(8167909) WEXITSTATUS 0 Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_617(8167909) done Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_619(8167911) WEXITSTATUS 0 Nov 05 09:28:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_619(8167911) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_812(8168104) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_812(8168104) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_597(8167889) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_597(8167889) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_984(8168276) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_984(8168276) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_983(8168275) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_983(8168275) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_982(8168274) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_982(8168274) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_949(8168241) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_949(8168241) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_606(8167898) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_606(8167898) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_950(8168242) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_950(8168242) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_869(8168161) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_869(8168161) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_868(8168160) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_868(8168160) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_773(8168065) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_773(8168065) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_772(8168064) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_772(8168064) done Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_771(8168063) WEXITSTATUS 0 Nov 05 09:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_771(8168063) done Nov 05 09:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_969(8168261) WEXITSTATUS 0 Nov 05 09:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_969(8168261) done Nov 05 09:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_968(8168260) WEXITSTATUS 0 Nov 05 09:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_968(8168260) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_916(8168208) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_916(8168208) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_917(8168209) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_917(8168209) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_918(8168210) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_918(8168210) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_690(8167982) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_690(8167982) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_691(8167983) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_691(8167983) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_688(8167980) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_688(8167980) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_689(8167981) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_689(8167981) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_899(8168191) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_899(8168191) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_901(8168193) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_901(8168193) done Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_898(8168190) WEXITSTATUS 0 Nov 05 09:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_898(8168190) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_583(8167875) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_583(8167875) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_642(8167934) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_642(8167934) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_640(8167932) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_640(8167932) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_638(8167930) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_638(8167930) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_586(8167878) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_586(8167878) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_585(8167877) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_585(8167877) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_582(8167874) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_582(8167874) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_994(8168286) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_994(8168286) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_991(8168283) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_991(8168283) done Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_993(8168285) WEXITSTATUS 0 Nov 05 09:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_993(8168285) done Nov 05 09:28:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_748(8168040) WEXITSTATUS 0 Nov 05 09:28:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_748(8168040) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_737(8168029) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_737(8168029) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_738(8168030) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_738(8168030) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_649(8167941) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_649(8167941) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_651(8167943) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_651(8167943) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_650(8167942) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_650(8167942) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_870(8168162) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_870(8168162) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_652(8167944) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_652(8167944) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_790(8168082) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_790(8168082) done Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_788(8168080) WEXITSTATUS 0 Nov 05 09:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_788(8168080) done Nov 05 09:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_973(8168265) WEXITSTATUS 0 Nov 05 09:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_973(8168265) done Nov 05 09:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_859(8168151) WEXITSTATUS 0 Nov 05 09:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_859(8168151) done Nov 05 09:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_861(8168153) WEXITSTATUS 0 Nov 05 09:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_861(8168153) done Nov 05 09:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_858(8168150) WEXITSTATUS 0 Nov 05 09:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_858(8168150) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_762(8168054) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_762(8168054) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_760(8168052) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_760(8168052) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_758(8168050) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_758(8168050) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_756(8168048) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_756(8168048) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_761(8168053) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_761(8168053) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_755(8168047) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_755(8168047) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_757(8168049) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_757(8168049) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_696(8167988) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_696(8167988) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_694(8167986) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_694(8167986) done Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_697(8167989) WEXITSTATUS 0 Nov 05 09:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_697(8167989) done Nov 05 09:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_803(8168095) WEXITSTATUS 0 Nov 05 09:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_803(8168095) done Nov 05 09:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1930(8163309) WEXITSTATUS 0 Nov 05 09:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1930(8163309) done Nov 05 09:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_866(8168158) WEXITSTATUS 0 Nov 05 09:28:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_866(8168158) done Nov 05 09:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_981(8168273) WEXITSTATUS 0 Nov 05 09:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_981(8168273) done Nov 05 09:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_978(8168270) WEXITSTATUS 0 Nov 05 09:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_978(8168270) done Nov 05 09:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_980(8168272) WEXITSTATUS 0 Nov 05 09:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_980(8168272) done Nov 05 09:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_931(8168223) WEXITSTATUS 0 Nov 05 09:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_931(8168223) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_605(8167897) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_605(8167897) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_602(8167894) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_602(8167894) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_603(8167895) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_603(8167895) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_781(8168073) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_781(8168073) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_786(8168078) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_786(8168078) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_996(8168288) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_996(8168288) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_997(8168289) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_997(8168289) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_643(8167935) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_643(8167935) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_644(8167936) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_644(8167936) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_647(8167939) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_647(8167939) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_645(8167937) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_645(8167937) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_646(8167938) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_646(8167938) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_569(8167861) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_569(8167861) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_564(8167856) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_564(8167856) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_571(8167863) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_571(8167863) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_566(8167858) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_566(8167858) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_563(8167855) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_563(8167855) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_565(8167857) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_565(8167857) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_567(8167859) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_567(8167859) done Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_562(8167854) WEXITSTATUS 0 Nov 05 09:28:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_562(8167854) done Nov 05 09:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_967(8168259) WEXITSTATUS 0 Nov 05 09:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_967(8168259) done Nov 05 09:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_632(8167924) WEXITSTATUS 0 Nov 05 09:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_632(8167924) done Nov 05 09:28:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_686(8167978) WEXITSTATUS 0 Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_686(8167978) done Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2823(8168316) NodeList=r909u13n01 #CPUs=1 Partition=day Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2824(8168317) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2825(8168318) NodeList=r815u23n03 #CPUs=1 Partition=day Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2826(8168319) NodeList=r816u03n01 #CPUs=1 Partition=day Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_572(8167864) WEXITSTATUS 0 Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_572(8167864) done Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_573(8167865) WEXITSTATUS 0 Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_573(8167865) done Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_726(8168018) WEXITSTATUS 0 Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_726(8168018) done Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_724(8168016) WEXITSTATUS 0 Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_724(8168016) done Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_729(8168021) WEXITSTATUS 0 Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_729(8168021) done Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_731(8168023) WEXITSTATUS 0 Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_731(8168023) done Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_727(8168019) WEXITSTATUS 0 Nov 05 09:28:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_727(8168019) done Nov 05 09:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_966(8168258) WEXITSTATUS 0 Nov 05 09:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_966(8168258) done Nov 05 09:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_965(8168257) WEXITSTATUS 0 Nov 05 09:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_965(8168257) done Nov 05 09:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_852(8168144) WEXITSTATUS 0 Nov 05 09:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_852(8168144) done Nov 05 09:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_611(8167903) WEXITSTATUS 0 Nov 05 09:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_611(8167903) done Nov 05 09:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_636(8167928) WEXITSTATUS 0 Nov 05 09:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_636(8167928) done Nov 05 09:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_634(8167926) WEXITSTATUS 0 Nov 05 09:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_634(8167926) done Nov 05 09:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_974(8168266) WEXITSTATUS 0 Nov 05 09:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_974(8168266) done Nov 05 09:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_578(8167870) WEXITSTATUS 0 Nov 05 09:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_578(8167870) done Nov 05 09:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_577(8167869) WEXITSTATUS 0 Nov 05 09:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_577(8167869) done Nov 05 09:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_679(8167971) WEXITSTATUS 0 Nov 05 09:28:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_679(8167971) done Nov 05 09:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_770(8168062) WEXITSTATUS 0 Nov 05 09:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_770(8168062) done Nov 05 09:28:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_855(8168147) WEXITSTATUS 0 Nov 05 09:28:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_855(8168147) done Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_942(8168234) WEXITSTATUS 0 Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_942(8168234) done Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_943(8168235) WEXITSTATUS 0 Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_943(8168235) done Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_946(8168238) WEXITSTATUS 0 Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_946(8168238) done Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_944(8168236) WEXITSTATUS 0 Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_944(8168236) done Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_945(8168237) WEXITSTATUS 0 Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_945(8168237) done Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_947(8168239) WEXITSTATUS 0 Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_947(8168239) done Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_940(8168232) WEXITSTATUS 0 Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_940(8168232) done Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_941(8168233) WEXITSTATUS 0 Nov 05 09:29:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_941(8168233) done Nov 05 09:29:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_939(8168231) WEXITSTATUS 0 Nov 05 09:29:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167267_939(8168231) done Nov 05 09:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7370(8166112) WEXITSTATUS 1 Nov 05 09:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7370(8166112) done Nov 05 09:29:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8168320 NodeList=(null) usec=54486 Nov 05 09:29:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168320 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7629(8167079) WEXITSTATUS 1 Nov 05 09:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7629(8167079) done Nov 05 09:29:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8078939 uid 20870 Nov 05 09:29:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1460(8138137) WEXITSTATUS 0 Nov 05 09:29:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1460(8138137) done Nov 05 09:29:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7493(8166466) WEXITSTATUS 1 Nov 05 09:29:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7493(8166466) done Nov 05 09:29:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168321 InitPrio=20028 usec=3984 Nov 05 09:30:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6321884 in day on r807u05n03 Nov 05 09:30:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r904u04n02 Nov 05 09:30:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7679(8168322) in scavenge on r815u03n12 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7680(8168323) in scavenge on r815u15n09 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7681(8168324) in scavenge on r815u15n12 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7682(8168325) in scavenge on r816u03n09 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7683(8168326) in scavenge on r816u03n10 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7684(8168327) in scavenge on r816u09n09 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7685(8168328) in scavenge on r816u23n07 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7686(8168329) in scavenge on r816u23n09 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7687(8168330) in scavenge on r816u23n09 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_450(8114717) in scavenge on r816u09n09 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_808(8115077) in scavenge on r816u23n09 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_879(8115148) in scavenge on r816u29n10 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_486(7572654) in scavenge on r805u27n03 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_700(7572869) in scavenge on r912u36n02 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3923(8167369) WEXITSTATUS 0 Nov 05 09:30:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3923(8167369) done Nov 05 09:30:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168331 InitPrio=16342 usec=6962 Nov 05 09:30:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2827(8168332) in scavenge on r801u06n03 Nov 05 09:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2828(8168333) in scavenge on r816u29n10 Nov 05 09:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2829(8168334) in scavenge on r816u29n10 Nov 05 09:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2830(8168335) in scavenge on r816u29n11 Nov 05 09:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2831(8168336) in scavenge on r806u31n01 Nov 05 09:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2832(8168337) in scavenge on r805u38n01 Nov 05 09:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2833(8168338) in scavenge on r904u13n01 Nov 05 09:30:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2834(8168339) in scavenge on r904u13n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:30:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:30:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:30:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 09:30:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 09:30:28 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 09:30:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3926(8168298) WEXITSTATUS 0 Nov 05 09:30:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3926(8168298) done Nov 05 09:30:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:30:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7592(8167007) WEXITSTATUS 1 Nov 05 09:30:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7592(8167007) done Nov 05 09:30:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:30:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7466(8166320) WEXITSTATUS 1 Nov 05 09:30:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7466(8166320) done Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162792 WEXITSTATUS 0 Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES cpu grp_used_tres_run_secs underflow, tried to remove 915088 seconds when only 914128 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES cpu grp_used_tres_run_secs underflow, tried to remove 915088 seconds when only 914128 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES mem grp_used_tres_run_secs underflow, tried to remove 1874100224 seconds when only 1872134144 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES mem grp_used_tres_run_secs underflow, tried to remove 1874100224 seconds when only 1872134144 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES node grp_used_tres_run_secs underflow, tried to remove 57193 seconds when only 57133 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES billing grp_used_tres_run_secs underflow, tried to remove 1715790 seconds when only 1713990 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 114386 seconds when only 114266 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES node grp_used_tres_run_secs underflow, tried to remove 57193 seconds when only 57133 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES cpu grp_used_tres_run_secs underflow, tried to remove 915088 seconds when only 914128 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES mem grp_used_tres_run_secs underflow, tried to remove 1874100224 seconds when only 1872134144 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES node grp_used_tres_run_secs underflow, tried to remove 57193 seconds when only 57133 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES billing grp_used_tres_run_secs underflow, tried to remove 1715790 seconds when only 1713990 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 114386 seconds when only 114266 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES billing grp_used_tres_run_secs underflow, tried to remove 1715790 seconds when only 1713990 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 9349 TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 114386 seconds when only 114266 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES cpu grp_used_tres_run_secs underflow, tried to remove 915088 seconds when only 914128 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES mem grp_used_tres_run_secs underflow, tried to remove 1874100224 seconds when only 1872134144 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES node grp_used_tres_run_secs underflow, tried to remove 57193 seconds when only 57133 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES billing grp_used_tres_run_secs underflow, tried to remove 1715790 seconds when only 1713990 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8162792: assoc 5491 TRES gres/gpu grp_used_tres_run_secs underflow, tried to remove 114386 seconds when only 114266 remained. Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162792 done Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_19(8165375) WEXITSTATUS 0 Nov 05 09:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_19(8165375) done Nov 05 09:30:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168340 InitPrio=13509 usec=2658 Nov 05 09:30:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168341 InitPrio=13509 usec=2741 Nov 05 09:30:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168342 InitPrio=13509 usec=1925 Nov 05 09:30:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168343 InitPrio=13509 usec=1978 Nov 05 09:30:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168344 InitPrio=13509 usec=1535 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168345 InitPrio=13509 usec=1543 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168346 InitPrio=13509 usec=1939 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168347 InitPrio=13509 usec=1506 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168348 InitPrio=13509 usec=1254 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168349 InitPrio=13509 usec=1854 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168350 InitPrio=13509 usec=1431 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168351 InitPrio=13509 usec=1612 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168352 InitPrio=13509 usec=1467 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168353 InitPrio=13509 usec=1624 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168354 InitPrio=13509 usec=2514 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168355 InitPrio=13509 usec=1506 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168356 InitPrio=13509 usec=1348 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168357 InitPrio=13509 usec=1287 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168358 InitPrio=13509 usec=1246 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168359 InitPrio=13509 usec=1281 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168360 InitPrio=13509 usec=1461 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168361 InitPrio=13509 usec=1381 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168362 InitPrio=13509 usec=1371 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168363 InitPrio=13509 usec=1465 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168364 InitPrio=13509 usec=1318 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168365 InitPrio=13509 usec=1251 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168366 InitPrio=13509 usec=1256 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168367 InitPrio=13509 usec=1237 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168368 InitPrio=13509 usec=1527 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168369 InitPrio=13509 usec=1504 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168370 InitPrio=13509 usec=1486 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168371 InitPrio=13509 usec=1259 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168372 InitPrio=13509 usec=1255 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168373 InitPrio=13509 usec=1261 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168374 InitPrio=13509 usec=1202 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168375 InitPrio=13509 usec=1226 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168376 InitPrio=13509 usec=1423 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168377 InitPrio=13509 usec=1326 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168378 InitPrio=13509 usec=1218 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168379 InitPrio=13509 usec=1232 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168380 InitPrio=13509 usec=1216 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168381 InitPrio=13509 usec=1202 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168382 InitPrio=13509 usec=1226 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168383 InitPrio=13509 usec=1476 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168384 InitPrio=13509 usec=1331 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168385 InitPrio=13509 usec=1222 Nov 05 09:30:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168386 InitPrio=13509 usec=1251 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5265(8158699) WEXITSTATUS 1 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5265(8158699) done Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168387 InitPrio=13509 usec=2120 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168388 InitPrio=13509 usec=2019 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168389 InitPrio=13509 usec=1952 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168390 InitPrio=13509 usec=1531 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168391 InitPrio=13509 usec=1461 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168392 InitPrio=13509 usec=1369 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168393 InitPrio=13509 usec=1395 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168394 InitPrio=13509 usec=1360 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168395 InitPrio=13509 usec=1369 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168396 InitPrio=13509 usec=1361 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168397 InitPrio=13509 usec=1319 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168398 InitPrio=13509 usec=1443 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168399 InitPrio=13509 usec=1383 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168400 InitPrio=13509 usec=1272 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168401 InitPrio=13509 usec=1261 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168402 InitPrio=13509 usec=1240 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168403 InitPrio=13509 usec=1259 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168404 InitPrio=13509 usec=1370 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168405 InitPrio=13509 usec=1305 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168406 InitPrio=13509 usec=1255 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168407 InitPrio=13509 usec=1237 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168408 InitPrio=13509 usec=1192 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168409 InitPrio=13509 usec=1229 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168410 InitPrio=13509 usec=1234 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168411 InitPrio=13509 usec=1255 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168412 InitPrio=13509 usec=1209 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168413 InitPrio=13509 usec=1255 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168414 InitPrio=13509 usec=2940 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168415 InitPrio=13509 usec=2444 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168416 InitPrio=13509 usec=1620 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168417 InitPrio=13509 usec=1467 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168418 InitPrio=13509 usec=1303 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168419 InitPrio=13509 usec=1277 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168420 InitPrio=13509 usec=1270 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168421 InitPrio=13509 usec=1260 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168422 InitPrio=13509 usec=1249 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168423 InitPrio=13509 usec=1237 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168424 InitPrio=13509 usec=1255 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168425 InitPrio=13509 usec=1251 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168426 InitPrio=13509 usec=1227 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168427 InitPrio=13509 usec=1567 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168428 InitPrio=13509 usec=1470 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168429 InitPrio=13509 usec=1256 Nov 05 09:30:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168430 InitPrio=13509 usec=1279 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168431 InitPrio=13509 usec=3129 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168432 InitPrio=13509 usec=1928 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168433 InitPrio=13509 usec=1672 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168434 InitPrio=13509 usec=2409 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168435 InitPrio=13509 usec=1607 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168436 InitPrio=13509 usec=1359 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168437 InitPrio=13509 usec=1339 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168438 InitPrio=13509 usec=1294 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168439 InitPrio=13509 usec=2892 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168440 InitPrio=13509 usec=2454 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168441 InitPrio=13509 usec=1479 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168442 InitPrio=13509 usec=1330 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168443 InitPrio=13509 usec=1272 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168444 InitPrio=13509 usec=1229 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168445 InitPrio=13509 usec=1265 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168446 InitPrio=13509 usec=1264 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168447 InitPrio=13509 usec=1213 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168448 InitPrio=13509 usec=1225 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168449 InitPrio=13509 usec=1241 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168450 InitPrio=13509 usec=1243 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168451 InitPrio=13509 usec=1231 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168452 InitPrio=13509 usec=1202 Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:30:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:30:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_20(8165376) WEXITSTATUS 0 Nov 05 09:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_20(8165376) done Nov 05 09:31:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7409(8166188) WEXITSTATUS 1 Nov 05 09:31:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7409(8166188) done Nov 05 09:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2640(8153027) WEXITSTATUS 0 Nov 05 09:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2640(8153027) done Nov 05 09:31:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168453 InitPrio=33918 usec=5693 Nov 05 09:31:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168453 NodeList=r904u04n02 #CPUs=1 Partition=day Nov 05 09:31:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6078(8163105) WEXITSTATUS 1 Nov 05 09:31:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6078(8163105) done Nov 05 09:31:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7682(8168325) WEXITSTATUS 1 Nov 05 09:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7682(8168325) done Nov 05 09:31:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168454 InitPrio=29041 usec=7443 Nov 05 09:31:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7670(8168306) WEXITSTATUS 1 Nov 05 09:31:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7670(8168306) done Nov 05 09:31:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:30 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:31:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2247(8165039) WEXITSTATUS 0 Nov 05 09:31:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2247(8165039) done Nov 05 09:31:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1939(8163399) WEXITSTATUS 0 Nov 05 09:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1939(8163399) done Nov 05 09:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_11(8158280) WEXITSTATUS 1 Nov 05 09:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158125_11(8158280) done Nov 05 09:31:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:31:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168454 NodeList=r909u10n02 #CPUs=4 Partition=day Nov 05 09:31:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7556(8166753) WEXITSTATUS 1 Nov 05 09:31:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7556(8166753) done Nov 05 09:32:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7342(8166046) WEXITSTATUS 1 Nov 05 09:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7342(8166046) done Nov 05 09:32:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_893(8128055) WEXITSTATUS 0 Nov 05 09:32:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_893(8128055) done Nov 05 09:32:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168455 InitPrio=59526 usec=4352 Nov 05 09:32:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7312(8165996) WEXITSTATUS 1 Nov 05 09:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7312(8165996) done Nov 05 09:32:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6223(8163375) WEXITSTATUS 1 Nov 05 09:32:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6223(8163375) done Nov 05 09:32:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:32:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:32:30 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8166000 Nov 05 09:32:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7496(8166478) WEXITSTATUS 1 Nov 05 09:32:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7496(8166478) done Nov 05 09:32:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7438(8166231) WEXITSTATUS 1 Nov 05 09:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7438(8166231) done Nov 05 09:32:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 09:32:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 09:32:37 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 09:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2178(8164818) WEXITSTATUS 0 Nov 05 09:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2178(8164818) done Nov 05 09:32:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_23(8165379) WEXITSTATUS 0 Nov 05 09:32:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_23(8165379) done Nov 05 09:32:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3924(8168296) WEXITSTATUS 0 Nov 05 09:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3924(8168296) done Nov 05 09:32:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:32:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3927(8168456) in gpu on r903u34n01 Nov 05 09:32:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3928(8168457) in gpu on r808u08n01 Nov 05 09:32:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3929(8168458) in gpu on r4513u29n01 Nov 05 09:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2835(8168459) in day on r805u02n01 Nov 05 09:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2836(8168460) in day on r909u29n03 Nov 05 09:32:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_492(7572660) WEXITSTATUS 0 Nov 05 09:32:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_492(7572660) done Nov 05 09:32:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_40(8165296) WEXITSTATUS 0 Nov 05 09:32:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_40(8165296) done Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7688(8168461) in scavenge on r802u04n03 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7689(8168462) in scavenge on r802u04n04 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7690(8168463) in scavenge on r816u03n09 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7691(8168464) in scavenge on r904u07n04 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7692(8168465) in scavenge on r913u13n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7693(8168466) in scavenge on r914u05n04 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7694(8168467) in scavenge on r909u10n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7695(8168468) in scavenge on r909u13n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7696(8168469) in scavenge on r805u11n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7697(8168470) in scavenge on r805u23n04 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7698(8168471) in scavenge on r805u25n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7699(8168472) in scavenge on r918u01n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7700(8168473) in scavenge on r918u01n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1122(7573291) in scavenge on r805u40n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_297(8114563) in scavenge on r805u40n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1063(7573232) in scavenge on r805u40n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_596(8114863) in scavenge on r806u37n01 Nov 05 09:32:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_608(8114875) in scavenge on r806u37n01 Nov 05 09:32:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166657 WEXITSTATUS 1 Nov 05 09:32:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166657 done Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168340 in scavenge on r907u01n02 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168341 in scavenge on r907u01n03 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168342 in scavenge on r907u01n04 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168343 in scavenge on r907u05n01 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168344 in scavenge on r907u05n02 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168345 in scavenge on r907u05n03 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168346 in scavenge on r907u05n04 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168347 in scavenge on r907u09n02 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168348 in scavenge on r802u14n03 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168349 in scavenge on r802u23n02 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168350 in scavenge on r802u30n04 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168351 in scavenge on r915u38n03 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168352 in scavenge on r917u13n04 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168353 in scavenge on r801u06n03 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168354 in scavenge on r801u09n03 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168355 in scavenge on r801u09n04 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168356 in scavenge on r801u27n02 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168357 in scavenge on r911u09n01 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168358 in scavenge on r911u09n02 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168359 in scavenge on r911u09n03 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168360 in scavenge on r911u13n01 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168361 in scavenge on r911u24n02 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168362 in scavenge on r911u28n01 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168363 in scavenge on r911u28n02 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168364 in scavenge on r912u05n04 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168365 in scavenge on r815u23n07 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168366 in scavenge on r815u29n09 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168367 in scavenge on r815u03n11 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168368 in scavenge on r815u15n09 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168369 in scavenge on r815u15n10 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168370 in scavenge on r816u23n11 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168371 in scavenge on r816u29n08 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168372 in scavenge on r816u29n11 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168373 in scavenge on r903u12n01 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168374 in scavenge on r911u32n01 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168375 in scavenge on r913u32n01 Nov 05 09:32:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168376 in scavenge on r913u35n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168377 in scavenge on r807u22n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168378 in scavenge on r4513u23n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168379 in scavenge on r4513u26n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168380 in scavenge on r902u07n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168381 in scavenge on r906u39n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168382 in scavenge on r919u01n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168383 in scavenge on r919u05n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168384 in scavenge on r919u32n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168385 in scavenge on r919u32n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168386 in scavenge on r805u06n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168387 in scavenge on r805u09n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168388 in scavenge on r805u09n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168389 in scavenge on r805u09n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168390 in scavenge on r805u14n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168391 in scavenge on r805u14n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168392 in scavenge on r805u14n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168393 in scavenge on r805u23n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168394 in scavenge on r805u25n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168395 in scavenge on r805u25n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168396 in scavenge on r805u27n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168397 in scavenge on r805u27n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168398 in scavenge on r912u28n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168399 in scavenge on r912u28n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168400 in scavenge on r912u32n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168401 in scavenge on r912u32n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168402 in scavenge on r913u01n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168403 in scavenge on r913u01n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168404 in scavenge on r913u05n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168405 in scavenge on r913u09n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168406 in scavenge on r913u13n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168407 in scavenge on r913u24n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168408 in scavenge on r913u28n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168409 in scavenge on r913u28n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168410 in scavenge on r913u28n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168411 in scavenge on r815u03n08 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168412 in scavenge on r815u09n06 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168413 in scavenge on r816u09n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168414 in scavenge on r816u09n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168415 in scavenge on r816u09n07 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168416 in scavenge on r816u15n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168417 in scavenge on r816u15n05 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168418 in scavenge on r816u15n07 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168419 in scavenge on r815u23n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168420 in scavenge on r815u23n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168421 in scavenge on r815u29n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168422 in scavenge on r815u29n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168423 in scavenge on r815u29n04 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168424 in scavenge on r815u35n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168425 in scavenge on r815u35n03 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168426 in scavenge on r805u40n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168427 in scavenge on r908u24n02 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168428 in scavenge on r806u35n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168429 in scavenge on r805u32n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168430 in scavenge on r805u34n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168431 in scavenge on r805u36n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168432 in scavenge on r805u38n01 Nov 05 09:33:00 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:112 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 09:33:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160526 WEXITSTATUS 0 Nov 05 09:33:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160526 done Nov 05 09:33:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8168474 NodeList=(null) usec=21446 Nov 05 09:33:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168475 InitPrio=10596 usec=2431 Nov 05 09:33:04 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168474 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168366 WEXITSTATUS 1 Nov 05 09:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168366 done Nov 05 09:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168380 WEXITSTATUS 1 Nov 05 09:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168380 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168378 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168378 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168365 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168365 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168369 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168369 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168425 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168425 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168367 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168367 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168371 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168371 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168379 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168379 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168418 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168418 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168372 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168372 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168411 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168411 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168370 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168370 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168413 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168413 done Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168417 WEXITSTATUS 1 Nov 05 09:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168417 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168416 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168416 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168421 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168421 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168368 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168368 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168422 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168422 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168412 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168412 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168344 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168344 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168347 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168347 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168345 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168345 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168346 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168346 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168415 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168415 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168424 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168424 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168341 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168341 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168342 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168342 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168420 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168420 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168340 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168340 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168423 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168423 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168419 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168419 done Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168343 WEXITSTATUS 1 Nov 05 09:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168343 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168414 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168414 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168385 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168385 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168349 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168349 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168383 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168383 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168374 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168374 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168373 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168373 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168352 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168352 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168386 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168386 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168348 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168348 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168359 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168359 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168382 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168382 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168353 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168353 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168398 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168398 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168354 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168354 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168356 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168356 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168364 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168364 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168410 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168410 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168355 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168355 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168392 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168392 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168377 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168377 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168391 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168391 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168360 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168360 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168408 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168408 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168405 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168405 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168395 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168395 done Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168388 WEXITSTATUS 1 Nov 05 09:33:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168388 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168357 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168357 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168393 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168393 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168362 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168362 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168384 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168384 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168387 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168387 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168351 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168351 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168375 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168375 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168407 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168407 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168402 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168402 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168376 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168376 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168403 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168403 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168358 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168358 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168397 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168397 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168394 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168394 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168390 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168390 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168389 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168389 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168396 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168396 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168381 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168381 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168404 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168404 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168427 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168427 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168409 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168409 done Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168399 WEXITSTATUS 1 Nov 05 09:33:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168399 done Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168430 WEXITSTATUS 1 Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168430 done Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168431 WEXITSTATUS 1 Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168431 done Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168429 WEXITSTATUS 1 Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168429 done Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168428 WEXITSTATUS 1 Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168428 done Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168406 WEXITSTATUS 1 Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168406 done Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168363 WEXITSTATUS 1 Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168363 done Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168350 WEXITSTATUS 1 Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168350 done Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168426 WEXITSTATUS 1 Nov 05 09:33:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168426 done Nov 05 09:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168432 WEXITSTATUS 1 Nov 05 09:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168432 done Nov 05 09:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168361 WEXITSTATUS 1 Nov 05 09:33:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168361 done Nov 05 09:33:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168476 InitPrio=10596 usec=3602 Nov 05 09:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168401 WEXITSTATUS 1 Nov 05 09:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168401 done Nov 05 09:33:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168477 InitPrio=10596 usec=3733 Nov 05 09:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167021 WEXITSTATUS 0 Nov 05 09:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167021 done Nov 05 09:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_16(8165372) WEXITSTATUS 0 Nov 05 09:33:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_16(8165372) done Nov 05 09:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168400 WEXITSTATUS 1 Nov 05 09:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168400 done Nov 05 09:33:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168478 InitPrio=10596 usec=5198 Nov 05 09:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7472(8166340) WEXITSTATUS 1 Nov 05 09:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7472(8166340) done Nov 05 09:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3929(8168458) WEXITSTATUS 0 Nov 05 09:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3929(8168458) done Nov 05 09:33:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168479 InitPrio=10596 usec=5045 Nov 05 09:33:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168480 InitPrio=10596 usec=3855 Nov 05 09:33:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168481 InitPrio=10596 usec=3841 Nov 05 09:33:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168482 InitPrio=10596 usec=4876 Nov 05 09:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3928(8168457) WEXITSTATUS 0 Nov 05 09:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3928(8168457) done Nov 05 09:33:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168483 InitPrio=10596 usec=3841 Nov 05 09:33:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168484 InitPrio=10596 usec=3745 Nov 05 09:33:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168485 InitPrio=10596 usec=3706 Nov 05 09:33:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168486 InitPrio=10596 usec=3914 Nov 05 09:33:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1937(8163397) WEXITSTATUS 0 Nov 05 09:33:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1937(8163397) done Nov 05 09:33:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168487 InitPrio=24840 usec=2284 Nov 05 09:33:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168487 NodeList=r903u01n02 #CPUs=4 Partition=pi_zhu Nov 05 09:33:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8168488 NodeList=(null) usec=26781 Nov 05 09:33:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_696(7572865) WEXITSTATUS 0 Nov 05 09:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_696(7572865) done Nov 05 09:33:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_17(8165373) WEXITSTATUS 0 Nov 05 09:33:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_17(8165373) done Nov 05 09:33:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168488 NodeList=r904u13n01 #CPUs=8 Partition=pi_ying_rex Nov 05 09:33:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:33:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6640(8164715) WEXITSTATUS 1 Nov 05 09:33:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6640(8164715) done Nov 05 09:34:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:34:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3918(8167088) WEXITSTATUS 0 Nov 05 09:34:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3918(8167088) done Nov 05 09:34:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:34:20 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:34:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1927(8163306) WEXITSTATUS 0 Nov 05 09:34:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1927(8163306) done Nov 05 09:34:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7644(8167112) WEXITSTATUS 1 Nov 05 09:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7644(8167112) done Nov 05 09:34:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160527 WEXITSTATUS 0 Nov 05 09:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160527 done Nov 05 09:34:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:34:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3921(8167251) WEXITSTATUS 0 Nov 05 09:34:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3921(8167251) done Nov 05 09:34:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:34:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166563 uid 25715 Nov 05 09:34:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:34:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144144 NodeList=r801u11n02,r801u25n04 #CPUs=72 Partition=scavenge Nov 05 09:35:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7647(8167252) WEXITSTATUS 1 Nov 05 09:35:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7647(8167252) done Nov 05 09:35:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7528(8166571) WEXITSTATUS 1 Nov 05 09:35:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7528(8166571) done Nov 05 09:35:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:35:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3930(8168489) in gpu on r903u34n01 Nov 05 09:35:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3931(8168490) in gpu on r808u08n01 Nov 05 09:35:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3932(8168491) in gpu on r808u11n01 Nov 05 09:35:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3933(8168492) in gpu on r4513u26n01 Nov 05 09:35:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2837(8168493) in day on r805u25n01 Nov 05 09:35:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2838(8168494) in day on r807u03n02 Nov 05 09:35:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2839(8168495) in day on r909u10n02 Nov 05 09:35:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u05n03 Nov 05 09:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_35(8165403) WEXITSTATUS 0 Nov 05 09:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_35(8165403) done Nov 05 09:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7701(8168496) in scavenge on r904u01n02 Nov 05 09:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7702(8168497) in scavenge on r816u29n11 Nov 05 09:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7703(8168498) in scavenge on r808u08n01 Nov 05 09:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7704(8168499) in scavenge on r907u34n01 Nov 05 09:35:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7705(8168500) in scavenge on r909u10n01 Nov 05 09:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_34(8165402) WEXITSTATUS 0 Nov 05 09:35:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_34(8165402) done Nov 05 09:35:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_37(8165405) WEXITSTATUS 0 Nov 05 09:35:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_37(8165405) done Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168433 in scavenge on r907u01n02 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168434 in scavenge on r907u01n03 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168435 in scavenge on r907u01n04 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168436 in scavenge on r907u05n01 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168437 in scavenge on r907u05n02 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168438 in scavenge on r907u05n03 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168439 in scavenge on r907u05n04 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168440 in scavenge on r907u09n02 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168441 in scavenge on r802u14n03 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168442 in scavenge on r802u23n02 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168443 in scavenge on r802u30n04 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168444 in scavenge on r915u38n03 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168445 in scavenge on r917u13n04 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168446 in scavenge on r801u06n03 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168447 in scavenge on r801u09n03 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168448 in scavenge on r801u09n04 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168449 in scavenge on r801u27n02 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168450 in scavenge on r911u09n01 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168451 in scavenge on r911u09n02 Nov 05 09:35:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168452 in scavenge on r911u09n03 Nov 05 09:35:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1916(8163250) WEXITSTATUS 0 Nov 05 09:35:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1916(8163250) done Nov 05 09:35:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3932(8168491) WEXITSTATUS 0 Nov 05 09:35:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3932(8168491) done Nov 05 09:35:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3914(8167023) WEXITSTATUS 0 Nov 05 09:35:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3914(8167023) done Nov 05 09:35:47 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8168501 NodeList=(null) usec=22907 Nov 05 09:35:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168501 NodeList=r914u30n01 #CPUs=1 Partition=education_gpu Nov 05 09:35:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:35:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168440 WEXITSTATUS 1 Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168440 done Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168437 WEXITSTATUS 1 Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168437 done Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168439 WEXITSTATUS 1 Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168439 done Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168438 WEXITSTATUS 1 Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168438 done Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168434 WEXITSTATUS 1 Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168434 done Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168435 WEXITSTATUS 1 Nov 05 09:35:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168435 done Nov 05 09:35:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168433 WEXITSTATUS 1 Nov 05 09:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168433 done Nov 05 09:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168441 WEXITSTATUS 1 Nov 05 09:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168441 done Nov 05 09:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168442 WEXITSTATUS 1 Nov 05 09:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168442 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168446 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168446 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168436 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168436 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168502 InitPrio=28349 usec=3434 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168452 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168452 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168447 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168447 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168448 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168448 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168445 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168445 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168449 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168449 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168444 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168444 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168450 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168450 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168451 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168451 done Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168443 WEXITSTATUS 1 Nov 05 09:35:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168443 done Nov 05 09:35:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1912(8163232) WEXITSTATUS 0 Nov 05 09:35:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1912(8163232) done Nov 05 09:35:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:35:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7430(8166219) WEXITSTATUS 1 Nov 05 09:35:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7430(8166219) done Nov 05 09:35:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:36:00 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1001(7573170) Nov 05 09:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168488 WEXITSTATUS 0 Nov 05 09:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168488 done Nov 05 09:36:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:36:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:36:19 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:36:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_4(8158164) WEXITSTATUS 1 Nov 05 09:36:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158124_4(8158164) done Nov 05 09:36:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_76(8135775) WEXITSTATUS 0 Nov 05 09:36:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_76(8135775) done Nov 05 09:36:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154695 WEXITSTATUS 0 Nov 05 09:36:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154695 done Nov 05 09:36:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160531 WEXITSTATUS 0 Nov 05 09:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160531 done Nov 05 09:36:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:36:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:37:02 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:37:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:37:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1944(8163404) WEXITSTATUS 0 Nov 05 09:37:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1944(8163404) done Nov 05 09:37:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:37:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7662(8167614) WEXITSTATUS 1 Nov 05 09:37:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7662(8167614) done Nov 05 09:37:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7698(8168471) WEXITSTATUS 1 Nov 05 09:37:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7698(8168471) done Nov 05 09:37:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:37:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7514(8166518) WEXITSTATUS 1 Nov 05 09:37:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7514(8166518) done Nov 05 09:37:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:37:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7663(8167615) WEXITSTATUS 1 Nov 05 09:37:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7663(8167615) done Nov 05 09:37:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:38:00 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2866(8128525) Nov 05 09:38:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:38:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168502 in gpu on r808u11n01 Nov 05 09:38:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3934(8168503) in gpu on r808u17n01 Nov 05 09:38:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3935(8168504) in gpu on r4513u29n01 Nov 05 09:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168331 in day on r913u28n01 Nov 05 09:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2840(8168505) in day on r807u03n04 Nov 05 09:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2841(8168506) in day on r913u05n01 Nov 05 09:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2842(8168507) in day on r918u05n03 Nov 05 09:38:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2843(8168508) in day on r815u03n07 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7706(8168509) in scavenge on r801u30n03 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7707(8168510) in scavenge on r907u32n01 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7708(8168511) in scavenge on r909u29n04 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7709(8168512) in scavenge on r805u06n04 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7710(8168513) in scavenge on r805u06n04 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7711(8168514) in scavenge on r805u25n01 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_645(7572814) in scavenge on r802u02n03 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_462(7572630) in scavenge on r904u13n01 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168331 WEXITSTATUS 0 Nov 05 09:38:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168331 done Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:38:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1909(8163192) WEXITSTATUS 0 Nov 05 09:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1909(8163192) done Nov 05 09:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168502 WEXITSTATUS 1 Nov 05 09:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168502 done Nov 05 09:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2213(8164959) WEXITSTATUS 0 Nov 05 09:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2213(8164959) done Nov 05 09:38:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_116(7572283) Nov 05 09:38:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2872(8128536) Nov 05 09:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_38(8165406) WEXITSTATUS 0 Nov 05 09:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_38(8165406) done Nov 05 09:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5372(8159265) WEXITSTATUS 1 Nov 05 09:38:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5372(8159265) done Nov 05 09:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7498(8166480) WEXITSTATUS 1 Nov 05 09:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7498(8166480) done Nov 05 09:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_79(8135693) WEXITSTATUS 0 Nov 05 09:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_79(8135693) done Nov 05 09:38:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168515 InitPrio=76232 usec=5981 Nov 05 09:38:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168515 NodeList=r815u09n10 #CPUs=6 Partition=pi_manohar Nov 05 09:38:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154693 WEXITSTATUS 0 Nov 05 09:38:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154693 done Nov 05 09:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_5(8165209) WEXITSTATUS 0 Nov 05 09:39:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_5(8165209) done Nov 05 09:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1920(8163276) WEXITSTATUS 0 Nov 05 09:39:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1920(8163276) done Nov 05 09:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7416(8166195) WEXITSTATUS 1 Nov 05 09:39:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7416(8166195) done Nov 05 09:39:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158124_2(8158162) Nov 05 09:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_11(8165247) WEXITSTATUS 0 Nov 05 09:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_11(8165247) done Nov 05 09:39:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3930(8168489) WEXITSTATUS 0 Nov 05 09:39:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3930(8168489) done Nov 05 09:39:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:39:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_34(8165290) WEXITSTATUS 0 Nov 05 09:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_34(8165290) done Nov 05 09:40:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:40:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1949(8163485) WEXITSTATUS 0 Nov 05 09:40:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1949(8163485) done Nov 05 09:40:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:40:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:40:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:40:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7401(8166171) WEXITSTATUS 1 Nov 05 09:40:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7401(8166171) done Nov 05 09:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3936(8168516) in gpu on r808u11n01 Nov 05 09:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2844(8168517) in day on r805u09n04 Nov 05 09:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2845(8168518) in day on r806u09n02 Nov 05 09:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2846(8168519) in day on r919u05n02 Nov 05 09:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2847(8168520) in day on r909u01n02 Nov 05 09:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2848(8168521) in day on r909u10n04 Nov 05 09:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2849(8168522) in day on r912u28n03 Nov 05 09:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2850(8168523) in day on r913u28n01 Nov 05 09:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u05n03 Nov 05 09:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168524 InitPrio=16342 usec=4327 Nov 05 09:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7712(8168525) in scavenge on r801u04n03 Nov 05 09:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7713(8168526) in scavenge on r802u02n04 Nov 05 09:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7714(8168527) in scavenge on r912u28n03 Nov 05 09:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7715(8168528) in scavenge on r913u01n02 Nov 05 09:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7716(8168529) in scavenge on r913u28n01 Nov 05 09:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1192(7573361) in scavenge on r802u02n04 Nov 05 09:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_26(8114292) in scavenge on r913u28n03 Nov 05 09:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168530 InitPrio=31874 usec=7518 Nov 05 09:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8168531 NodeList=(null) usec=24516 Nov 05 09:41:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168530 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:41:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168531 NodeList=r903u34n01 #CPUs=1 Partition=education_gpu Nov 05 09:41:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168532 InitPrio=16342 usec=3170 Nov 05 09:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7403(8166173) WEXITSTATUS 1 Nov 05 09:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7403(8166173) done Nov 05 09:41:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168533 InitPrio=16342 usec=5807 Nov 05 09:41:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168534 InitPrio=16342 usec=4134 Nov 05 09:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6934(8165217) WEXITSTATUS 1 Nov 05 09:41:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6934(8165217) done Nov 05 09:41:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_894(7573063) Nov 05 09:41:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_943(7573112) Nov 05 09:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7434(8166227) WEXITSTATUS 1 Nov 05 09:41:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7434(8166227) done Nov 05 09:41:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7550(8166747) WEXITSTATUS 1 Nov 05 09:41:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7550(8166747) done Nov 05 09:42:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:42:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2560(8152827) WEXITSTATUS 0 Nov 05 09:42:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2560(8152827) done Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:42:21 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:42:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7709(8168512) WEXITSTATUS 1 Nov 05 09:42:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7709(8168512) done Nov 05 09:42:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8167075 uid 23627 Nov 05 09:42:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168535 InitPrio=34706 usec=3715 Nov 05 09:42:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168535 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:42:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1999(8164216) WEXITSTATUS 0 Nov 05 09:42:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1999(8164216) done Nov 05 09:42:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6664(8164750) WEXITSTATUS 1 Nov 05 09:42:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6664(8164750) done Nov 05 09:42:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_41(8165297) WEXITSTATUS 0 Nov 05 09:42:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_41(8165297) done Nov 05 09:43:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7146(8165741) WEXITSTATUS 1 Nov 05 09:43:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7146(8165741) done Nov 05 09:43:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7513(8166517) WEXITSTATUS 1 Nov 05 09:43:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7513(8166517) done Nov 05 09:43:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168536 InitPrio=16342 usec=4235 Nov 05 09:43:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168537 InitPrio=57385 usec=7292 Nov 05 09:43:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168537 NodeList=r915u38n03 #CPUs=1 Partition=pi_lora Nov 05 09:43:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 09:43:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168524 in day on r904u04n02 Nov 05 09:43:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168532 in day on r806u04n01 Nov 05 09:43:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168533 in day on r806u04n01 Nov 05 09:43:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168534 in day on r806u04n01 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168538 InitPrio=16342 usec=4542 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168539 InitPrio=16342 usec=5625 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168532 WEXITSTATUS 0 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168532 done Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168540 InitPrio=16342 usec=5473 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168524 WEXITSTATUS 0 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168524 done Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168541 InitPrio=16342 usec=4652 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168533 WEXITSTATUS 0 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168533 done Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168534 WEXITSTATUS 0 Nov 05 09:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168534 done Nov 05 09:43:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168542 InitPrio=16358 usec=5096 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168542 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7717(8168543) in scavenge on r802u30n04 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7718(8168544) in scavenge on r915u32n01 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7719(8168545) in scavenge on r815u09n11 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7720(8168546) in scavenge on r816u23n11 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7721(8168547) in scavenge on r816u15n11 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7722(8168548) in scavenge on r816u15n12 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7723(8168549) in scavenge on r805u06n04 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7724(8168550) in scavenge on r807u11n01 Nov 05 09:43:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_172(7572340) in scavenge on r816u35n03 Nov 05 09:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2851(8168551) in scavenge on r816u35n10 Nov 05 09:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2852(8168552) in scavenge on r805u40n01 Nov 05 09:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2853(8168553) in scavenge on r805u40n01 Nov 05 09:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2854(8168554) in scavenge on r805u40n01 Nov 05 09:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2855(8168555) in scavenge on r805u40n01 Nov 05 09:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2856(8168556) in scavenge on r806u31n01 Nov 05 09:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1948(8163484) WEXITSTATUS 0 Nov 05 09:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1948(8163484) done Nov 05 09:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7673(8168309) WEXITSTATUS 1 Nov 05 09:44:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7673(8168309) done Nov 05 09:44:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2886(8128570) Nov 05 09:44:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2887(8128571) Nov 05 09:44:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2910(8128594) Nov 05 09:44:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2926(8128610) Nov 05 09:44:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2959(8128643) Nov 05 09:44:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168557 InitPrio=28268 usec=3297 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:44:22 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:44:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4307(8154073) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:44:31 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4636(8156228) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:44:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4636(8156228) Nov 05 09:44:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4307(8154073) Nov 05 09:44:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7690(8168463) WEXITSTATUS 1 Nov 05 09:44:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7690(8168463) done Nov 05 09:44:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8168558 NodeList=(null) usec=37924 Nov 05 09:44:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168558 NodeList=r806u23n03 #CPUs=4 Partition=devel Nov 05 09:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7468(8166336) WEXITSTATUS 1 Nov 05 09:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7468(8166336) done Nov 05 09:45:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8168559 NodeList=(null) usec=19107 Nov 05 09:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_462(7572630) WEXITSTATUS 0 Nov 05 09:45:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_462(7572630) done Nov 05 09:45:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3935(8168504) WEXITSTATUS 0 Nov 05 09:45:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3935(8168504) done Nov 05 09:45:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168559 NodeList=r904u13n01 #CPUs=8 Partition=pi_ying_rex Nov 05 09:45:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2603(8152870) WEXITSTATUS 0 Nov 05 09:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2603(8152870) done Nov 05 09:45:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3936(8168516) WEXITSTATUS 0 Nov 05 09:45:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3936(8168516) done Nov 05 09:45:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7571(8166809) WEXITSTATUS 1 Nov 05 09:45:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7571(8166809) done Nov 05 09:45:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_25(8165381) WEXITSTATUS 0 Nov 05 09:45:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_25(8165381) done Nov 05 09:45:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168560 InitPrio=16358 usec=4158 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168561 InitPrio=16358 usec=3760 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168562 InitPrio=16358 usec=3172 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168563 InitPrio=16358 usec=2089 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168564 InitPrio=16358 usec=1997 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168565 InitPrio=16358 usec=2284 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168566 InitPrio=16358 usec=1983 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168567 InitPrio=16358 usec=1930 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168568 InitPrio=16358 usec=1937 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168569 InitPrio=16358 usec=1960 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168570 InitPrio=16358 usec=2006 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168571 InitPrio=16358 usec=1879 Nov 05 09:45:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168572 InitPrio=16358 usec=6064 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=6676680_1(6676841) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=6676680_6(6676680) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6549(8164502) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168573 InitPrio=16358 usec=2070 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168574 InitPrio=16358 usec=1996 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168575 InitPrio=16358 usec=1864 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168576 InitPrio=16358 usec=1831 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168577 InitPrio=16358 usec=1820 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168578 InitPrio=16358 usec=1873 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168579 InitPrio=16358 usec=1822 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168580 InitPrio=16358 usec=2313 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168581 InitPrio=16358 usec=2222 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168582 InitPrio=16358 usec=4358 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168583 InitPrio=16358 usec=3157 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168584 InitPrio=16358 usec=2003 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168585 InitPrio=16358 usec=1857 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168586 InitPrio=16358 usec=1904 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168587 InitPrio=16358 usec=1862 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168588 InitPrio=16358 usec=1856 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168589 InitPrio=16358 usec=1860 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168590 InitPrio=16358 usec=1822 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168591 InitPrio=16358 usec=3480 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168592 InitPrio=16358 usec=1983 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168593 InitPrio=16358 usec=1830 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168594 InitPrio=16358 usec=5244 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168595 InitPrio=16358 usec=4100 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168596 InitPrio=16358 usec=2723 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168597 InitPrio=16358 usec=2079 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168598 InitPrio=16358 usec=2018 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168599 InitPrio=16358 usec=1899 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168600 InitPrio=16358 usec=1906 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168601 InitPrio=16358 usec=1954 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168602 InitPrio=16358 usec=1889 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168603 InitPrio=16358 usec=1836 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168604 InitPrio=16358 usec=2842 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168605 InitPrio=16358 usec=2851 Nov 05 09:45:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168606 InitPrio=16358 usec=2414 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168607 InitPrio=16358 usec=4677 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168608 InitPrio=16358 usec=4075 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168609 InitPrio=16358 usec=3484 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168610 InitPrio=16358 usec=2387 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168611 InitPrio=16358 usec=2031 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168612 InitPrio=16358 usec=1963 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168613 InitPrio=16358 usec=1995 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168614 InitPrio=16358 usec=1937 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168615 InitPrio=16358 usec=5932 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168616 InitPrio=16358 usec=4227 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168617 InitPrio=16358 usec=2985 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168618 InitPrio=16358 usec=2342 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168619 InitPrio=16358 usec=2082 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168620 InitPrio=16358 usec=2088 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168621 InitPrio=16358 usec=1995 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168622 InitPrio=16358 usec=1976 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168623 InitPrio=16358 usec=1936 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168624 InitPrio=16358 usec=1886 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168625 InitPrio=16358 usec=2027 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168626 InitPrio=16358 usec=2495 Nov 05 09:45:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168627 InitPrio=16358 usec=2314 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6549(8164502) Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168628 InitPrio=16358 usec=4487 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6676680_6(6676680) Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6676680_1(6676841) Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168629 InitPrio=16358 usec=3774 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168630 InitPrio=16358 usec=4603 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168631 InitPrio=16358 usec=3263 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168632 InitPrio=16358 usec=3628 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168633 InitPrio=16358 usec=2577 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168634 InitPrio=16358 usec=3045 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168635 InitPrio=16358 usec=2329 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168636 InitPrio=16358 usec=2193 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168637 InitPrio=16358 usec=2086 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168638 InitPrio=16358 usec=2979 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168639 InitPrio=16358 usec=2511 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168640 InitPrio=16358 usec=2085 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168641 InitPrio=16358 usec=1969 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168642 InitPrio=16358 usec=1888 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168643 InitPrio=16358 usec=1901 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168644 InitPrio=16358 usec=1889 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168645 InitPrio=16358 usec=1875 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168646 InitPrio=16358 usec=1927 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168647 InitPrio=16358 usec=1929 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168648 InitPrio=16358 usec=1880 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168649 InitPrio=16358 usec=1835 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168650 InitPrio=16358 usec=2194 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168651 InitPrio=16358 usec=1863 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168652 InitPrio=16358 usec=1862 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168653 InitPrio=16358 usec=3364 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168654 InitPrio=16358 usec=2198 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168655 InitPrio=16358 usec=1946 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168656 InitPrio=16358 usec=1912 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168657 InitPrio=16358 usec=1955 Nov 05 09:45:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168658 InitPrio=16358 usec=1904 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168659 InitPrio=16358 usec=2048 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168660 InitPrio=16358 usec=3472 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168661 InitPrio=16358 usec=2916 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168662 InitPrio=16358 usec=2030 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168663 InitPrio=16358 usec=2042 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168664 InitPrio=16358 usec=2163 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168665 InitPrio=16358 usec=1954 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168666 InitPrio=16358 usec=1987 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168667 InitPrio=16358 usec=1925 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168668 InitPrio=16358 usec=2192 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168669 InitPrio=16358 usec=2112 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168670 InitPrio=16358 usec=2198 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168671 InitPrio=16358 usec=2152 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168672 InitPrio=16358 usec=2103 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168673 InitPrio=16358 usec=2406 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168674 InitPrio=16358 usec=2159 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168675 InitPrio=16358 usec=5247 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168676 InitPrio=16358 usec=3575 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168677 InitPrio=16358 usec=2579 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168678 InitPrio=16358 usec=3812 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168679 InitPrio=16358 usec=2838 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168680 InitPrio=16358 usec=2648 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168681 InitPrio=16358 usec=2228 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168682 InitPrio=16358 usec=2082 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168683 InitPrio=16358 usec=1952 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168684 InitPrio=16358 usec=1976 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168685 InitPrio=16358 usec=1927 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168686 InitPrio=16358 usec=1984 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168687 InitPrio=16358 usec=2074 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168688 InitPrio=16358 usec=1903 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168689 InitPrio=16358 usec=1906 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168690 InitPrio=16358 usec=2029 Nov 05 09:45:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168691 InitPrio=16358 usec=1990 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168692 InitPrio=16358 usec=4033 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168693 InitPrio=16358 usec=3910 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168694 InitPrio=16358 usec=2494 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168695 InitPrio=16358 usec=5228 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168696 InitPrio=16358 usec=4285 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168697 InitPrio=16358 usec=2467 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168698 InitPrio=16358 usec=2032 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168699 InitPrio=16358 usec=3834 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168700 InitPrio=16358 usec=2315 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168701 InitPrio=16358 usec=1993 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168702 InitPrio=16358 usec=1916 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168703 InitPrio=16358 usec=2535 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168704 InitPrio=16358 usec=2511 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168705 InitPrio=16358 usec=2283 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168706 InitPrio=16358 usec=2033 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168707 InitPrio=16358 usec=2031 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168708 InitPrio=16358 usec=1885 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168709 InitPrio=16358 usec=2053 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168710 InitPrio=16358 usec=1924 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168711 InitPrio=16358 usec=1880 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168712 InitPrio=16358 usec=1958 Nov 05 09:45:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168713 InitPrio=16358 usec=1920 Nov 05 09:45:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_9(8165336) WEXITSTATUS 0 Nov 05 09:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_9(8165336) done Nov 05 09:45:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:45:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168557 NodeList=r815u35n07 #CPUs=4 Partition=gpu_devel Nov 05 09:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_27(8165383) WEXITSTATUS 0 Nov 05 09:46:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_27(8165383) done Nov 05 09:46:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3937(8168714) in gpu on r808u11n01 Nov 05 09:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3938(8168715) in gpu on r4513u29n01 Nov 05 09:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_778(7572947) WEXITSTATUS 0 Nov 05 09:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_778(7572947) done Nov 05 09:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_29(8165385) WEXITSTATUS 0 Nov 05 09:46:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_29(8165385) done Nov 05 09:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u05n03 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7725(8168716) in scavenge on r802u02n02 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7726(8168717) in scavenge on r815u09n12 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7727(8168718) in scavenge on r816u03n09 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7728(8168719) in scavenge on r919u01n04 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7729(8168720) in scavenge on r919u05n03 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7730(8168721) in scavenge on r919u05n03 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7731(8168722) in scavenge on r815u29n03 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7732(8168723) in scavenge on r815u29n03 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7733(8168724) in scavenge on r815u29n03 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7734(8168725) in scavenge on r815u35n02 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7735(8168726) in scavenge on r816u03n04 Nov 05 09:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7736(8168727) in scavenge on r816u15n06 Nov 05 09:46:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7693(8168466) WEXITSTATUS 1 Nov 05 09:46:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7693(8168466) done Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:46:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:46:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2961(8128754) Nov 05 09:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7523(8166542) WEXITSTATUS 1 Nov 05 09:46:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7523(8166542) done Nov 05 09:46:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7728(8168719) WEXITSTATUS 1 Nov 05 09:46:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7728(8168719) done Nov 05 09:47:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6160(8163207) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:47:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6082(8163109) WEXITSTATUS 1 Nov 05 09:47:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6082(8163109) done Nov 05 09:47:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6160(8163207) Nov 05 09:47:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7605(8167034) WEXITSTATUS 1 Nov 05 09:47:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7605(8167034) done Nov 05 09:47:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168728 InitPrio=46697 usec=7025 Nov 05 09:47:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168728 NodeList=r903u37n01 #CPUs=1 Partition=gpu_devel Nov 05 09:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2571(8127764) WEXITSTATUS 1 Nov 05 09:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2571(8127764) done Nov 05 09:47:32 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:47:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3934(8168503) WEXITSTATUS 0 Nov 05 09:47:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3934(8168503) done Nov 05 09:47:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:47:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7536(8166668) WEXITSTATUS 1 Nov 05 09:47:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7536(8166668) done Nov 05 09:47:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:47:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168659 uid 21920 Nov 05 09:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166735 WEXITSTATUS 0 Nov 05 09:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166735 done Nov 05 09:47:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7022(8165447) WEXITSTATUS 1 Nov 05 09:47:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7022(8165447) done Nov 05 09:47:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:47:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:47:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7700(8168473) WEXITSTATUS 1 Nov 05 09:47:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7700(8168473) done Nov 05 09:47:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:47:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168657 uid 21920 Nov 05 09:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159215 WEXITSTATUS 0 Nov 05 09:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159215 done Nov 05 09:47:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7687(8168330) WEXITSTATUS 1 Nov 05 09:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7687(8168330) done Nov 05 09:48:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5849(8162820) WEXITSTATUS 1 Nov 05 09:48:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5849(8162820) done Nov 05 09:48:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_48(8165442) WEXITSTATUS 0 Nov 05 09:48:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_48(8165442) done Nov 05 09:48:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168656 uid 21920 Nov 05 09:48:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168729 InitPrio=10821 usec=2633 Nov 05 09:48:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168730 InitPrio=10821 usec=4575 Nov 05 09:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168294 WEXITSTATUS 0 Nov 05 09:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168294 done Nov 05 09:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7566(8166794) WEXITSTATUS 1 Nov 05 09:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7566(8166794) done Nov 05 09:48:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168731 InitPrio=13509 usec=1955 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168732 InitPrio=13509 usec=1926 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168733 InitPrio=13509 usec=1713 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168734 InitPrio=13509 usec=1680 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168735 InitPrio=13509 usec=1328 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168736 InitPrio=13509 usec=1289 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168737 InitPrio=13509 usec=2535 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168738 InitPrio=13509 usec=1532 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168739 InitPrio=13509 usec=1936 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168740 InitPrio=13509 usec=1528 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168741 InitPrio=13509 usec=1487 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168742 InitPrio=13509 usec=1520 Nov 05 09:48:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168743 InitPrio=13509 usec=1577 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168744 InitPrio=13509 usec=1465 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168745 InitPrio=13509 usec=1995 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168746 InitPrio=13509 usec=1479 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168747 InitPrio=13509 usec=1341 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168748 InitPrio=13509 usec=1940 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168749 InitPrio=13509 usec=1435 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168750 InitPrio=13509 usec=1507 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168751 InitPrio=13509 usec=1304 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168752 InitPrio=13509 usec=1335 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168753 InitPrio=13509 usec=1264 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168754 InitPrio=13509 usec=1372 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168755 InitPrio=13509 usec=1350 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168756 InitPrio=13509 usec=1461 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168757 InitPrio=13509 usec=1315 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168758 InitPrio=13509 usec=2725 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168759 InitPrio=13509 usec=2354 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168760 InitPrio=13509 usec=1531 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168761 InitPrio=13509 usec=1274 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168762 InitPrio=13509 usec=1330 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168763 InitPrio=13509 usec=2106 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168764 InitPrio=13509 usec=1444 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168765 InitPrio=13509 usec=1265 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168766 InitPrio=13509 usec=1903 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168767 InitPrio=13509 usec=1350 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168768 InitPrio=13509 usec=1275 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168769 InitPrio=13509 usec=1244 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168770 InitPrio=13509 usec=1250 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168771 InitPrio=13509 usec=1681 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168772 InitPrio=13509 usec=1687 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168773 InitPrio=13509 usec=1958 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168774 InitPrio=13509 usec=1469 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168775 InitPrio=13509 usec=1517 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168776 InitPrio=13509 usec=1362 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168777 InitPrio=13509 usec=1243 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168778 InitPrio=13509 usec=1263 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168779 InitPrio=13509 usec=1252 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168780 InitPrio=13509 usec=1203 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168781 InitPrio=13509 usec=1256 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168782 InitPrio=13509 usec=1339 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168783 InitPrio=13509 usec=1303 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168784 InitPrio=13509 usec=1261 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168785 InitPrio=13509 usec=1593 Nov 05 09:48:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168786 InitPrio=13509 usec=1397 Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168787 InitPrio=13509 usec=2200 Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168788 InitPrio=13509 usec=1930 Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168789 InitPrio=13509 usec=1950 Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168790 InitPrio=13509 usec=1660 Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 09:48:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168658 uid 21920 Nov 05 09:48:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7426(8166215) WEXITSTATUS 1 Nov 05 09:48:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7426(8166215) done Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:48:28 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:48:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162666 WEXITSTATUS 0 Nov 05 09:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8162666 done Nov 05 09:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7484(8166441) WEXITSTATUS 1 Nov 05 09:48:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7484(8166441) done Nov 05 09:48:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168652 uid 21920 Nov 05 09:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3927(8168456) WEXITSTATUS 0 Nov 05 09:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3927(8168456) done Nov 05 09:48:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168653 uid 21920 Nov 05 09:48:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7518(8166529) WEXITSTATUS 1 Nov 05 09:48:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7518(8166529) done Nov 05 09:48:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168791 InitPrio=10821 usec=2147 Nov 05 09:48:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168792 InitPrio=10821 usec=3321 Nov 05 09:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168292 WEXITSTATUS 0 Nov 05 09:48:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168292 done Nov 05 09:48:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168654 uid 21920 Nov 05 09:48:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:53 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 09:48:53 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 09:48:54 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 09:48:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 09:48:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168793 InitPrio=28349 usec=3613 Nov 05 09:48:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:56 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_1(8168794) in pi_ohern on r801u32n02 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_2(8168795) in pi_ohern on r917u13n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_3(8168796) in pi_ohern on r917u13n03 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_4(8168797) in pi_ohern on r917u13n03 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_5(8168798) in pi_ohern on r917u13n04 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_6(8168799) in pi_ohern on r917u13n04 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_7(8168800) in pi_ohern on r917u13n04 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_8(8168801) in pi_ohern on r917u13n04 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_9(8168802) in pi_ohern on r917u13n04 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_10(8168803) in pi_ohern on r917u13n04 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_11(8168804) in pi_ohern on r917u29n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_12(8168805) in pi_ohern on r917u29n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_13(8168806) in pi_ohern on r917u29n02 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_14(8168807) in pi_ohern on r917u29n02 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_15(8168808) in pi_ohern on r917u29n04 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_16(8168809) in pi_ohern on r917u29n04 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_17(8168810) in pi_ohern on r815u03n11 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_18(8168811) in pi_ohern on r815u03n11 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_19(8168812) in pi_ohern on r815u03n11 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_20(8168813) in pi_ohern on r815u03n11 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_21(8168814) in pi_ohern on r815u03n12 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_22(8168815) in pi_ohern on r815u03n12 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_23(8168816) in pi_ohern on r911u32n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_24(8168817) in pi_ohern on r911u32n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_25(8168818) in pi_ohern on r911u32n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_26(8168819) in pi_ohern on r911u32n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_27(8168820) in pi_ohern on r807u22n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_28(8168821) in pi_ohern on r807u22n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_29(8168822) in pi_ohern on r807u22n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_30(8168823) in pi_ohern on r807u22n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_31(8168824) in pi_ohern on r807u24n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_32(8168825) in pi_ohern on r913u32n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_33(8168826) in pi_ohern on r913u32n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_34(8168827) in pi_ohern on r913u32n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_35(8168828) in pi_ohern on r913u32n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_36(8168829) in pi_ohern on r913u35n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_37(8168830) in pi_ohern on r913u35n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_38(8168831) in pi_ohern on r913u35n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_39(8168832) in pi_ohern on r913u35n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_40(8168833) in pi_ohern on r903u12n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_41(8168834) in pi_ohern on r903u12n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_42(8168835) in pi_ohern on r903u12n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_43(8168836) in pi_ohern on r903u12n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_44(8168837) in pi_ohern on r903u24n01 Nov 05 09:48:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_45(8168838) in pi_ohern on r903u26n01 Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168793 in gpu on r903u34n01 Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1125(7570787) WEXITSTATUS 0 Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1125(7570787) done Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_992(8115261) WEXITSTATUS 0 Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_992(8115261) done Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_648(7571613) WEXITSTATUS 0 Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_648(7571613) done Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_831(7573000) WEXITSTATUS 0 Nov 05 09:48:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_831(7573000) done Nov 05 09:48:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:48:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3939(8168839) in gpu on r808u17n01 Nov 05 09:48:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8154978 in day on r806u04n01 Nov 05 09:48:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168560 in day on r805u06n03 Nov 05 09:48:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168536 in day on r805u25n04 Nov 05 09:49:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6676680_1(6676841) in gpu on r4513u20n01 Nov 05 09:49:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168655 uid 21920 Nov 05 09:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_469(8114736) WEXITSTATUS 0 Nov 05 09:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_469(8114736) done Nov 05 09:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_540(7571505) WEXITSTATUS 0 Nov 05 09:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_540(7571505) done Nov 05 09:49:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3011(8128912) Nov 05 09:49:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3931(8168490) WEXITSTATUS 0 Nov 05 09:49:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3931(8168490) done Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7737(8168840) in scavenge on r908u28n02 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7738(8168841) in scavenge on r911u05n02 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7739(8168842) in scavenge on r816u23n09 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7740(8168843) in scavenge on r816u23n11 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7741(8168844) in scavenge on r816u23n12 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7742(8168845) in scavenge on r805u04n03 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7743(8168846) in scavenge on r806u23n01 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7744(8168847) in scavenge on r807u11n04 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7745(8168848) in scavenge on r904u04n03 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7746(8168849) in scavenge on r913u24n02 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7747(8168850) in scavenge on r914u05n04 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7748(8168851) in scavenge on r909u01n04 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7749(8168852) in scavenge on r909u04n02 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7750(8168853) in scavenge on r918u01n01 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7751(8168854) in scavenge on r918u01n02 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7752(8168855) in scavenge on r918u01n04 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7753(8168856) in scavenge on r918u01n04 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7754(8168857) in scavenge on r919u01n03 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_800(8115069) in scavenge on r918u01n04 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_354(7572522) in scavenge on r919u32n03 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_625(8116098) in scavenge on r919u32n03 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_949(8116423) in scavenge on r815u29n05 Nov 05 09:49:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1137(7573306) in scavenge on r805u34n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168731 in scavenge on r907u01n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168732 in scavenge on r907u01n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168733 in scavenge on r907u01n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168734 in scavenge on r907u05n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168735 in scavenge on r907u05n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168736 in scavenge on r907u05n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168737 in scavenge on r907u05n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168738 in scavenge on r907u09n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168739 in scavenge on r907u09n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168740 in scavenge on r915u38n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168741 in scavenge on r802u14n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168742 in scavenge on r802u23n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168743 in scavenge on r802u30n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168744 in scavenge on r801u06n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168745 in scavenge on r801u09n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168746 in scavenge on r801u09n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168747 in scavenge on r801u27n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168748 in scavenge on r911u09n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168749 in scavenge on r911u09n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168750 in scavenge on r911u09n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168751 in scavenge on r911u13n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168752 in scavenge on r911u24n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168753 in scavenge on r911u28n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168754 in scavenge on r911u28n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168755 in scavenge on r912u05n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168756 in scavenge on r815u15n09 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168757 in scavenge on r815u15n10 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168758 in scavenge on r815u23n07 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168759 in scavenge on r815u29n09 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168760 in scavenge on r816u23n11 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168761 in scavenge on r816u29n08 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168762 in scavenge on r816u29n11 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168763 in scavenge on r4513u23n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168764 in scavenge on r4513u26n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168765 in scavenge on r902u07n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168766 in scavenge on r906u39n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168767 in scavenge on r919u01n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168768 in scavenge on r919u32n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168769 in scavenge on r805u06n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168770 in scavenge on r805u09n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168771 in scavenge on r805u09n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168772 in scavenge on r805u09n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168773 in scavenge on r805u14n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168774 in scavenge on r805u14n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168775 in scavenge on r805u14n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168776 in scavenge on r805u23n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168777 in scavenge on r805u25n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168778 in scavenge on r805u27n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168779 in scavenge on r805u27n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168780 in scavenge on r912u28n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168781 in scavenge on r912u28n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168782 in scavenge on r912u28n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168783 in scavenge on r912u32n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168784 in scavenge on r912u32n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168785 in scavenge on r913u01n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168786 in scavenge on r913u09n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168787 in scavenge on r913u24n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168788 in scavenge on r913u28n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168789 in scavenge on r913u28n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168790 in scavenge on r913u28n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_1(8168858) in scavenge on r908u28n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_2(8168859) in scavenge on r908u33n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_3(8168860) in scavenge on r908u33n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_4(8168861) in scavenge on r908u33n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_5(8168862) in scavenge on r908u33n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_6(8168863) in scavenge on r907u05n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_7(8168864) in scavenge on r908u35n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_8(8168865) in scavenge on r908u35n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_9(8168866) in scavenge on r908u35n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168729_10(8168729) in scavenge on r908u35n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_1(8168867) in scavenge on r908u37n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_2(8168868) in scavenge on r908u37n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_3(8168869) in scavenge on r908u37n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_4(8168870) in scavenge on r908u39n01 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_5(8168871) in scavenge on r908u39n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_6(8168872) in scavenge on r908u39n03 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_7(8168873) in scavenge on r908u39n04 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_8(8168874) in scavenge on r903u01n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_9(8168875) in scavenge on r903u01n02 Nov 05 09:49:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168791_10(8168791) in scavenge on r903u01n02 Nov 05 09:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3925(8168297) WEXITSTATUS 0 Nov 05 09:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3925(8168297) done Nov 05 09:49:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168672 uid 21920 Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168763 WEXITSTATUS 1 Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168763 done Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168765 WEXITSTATUS 1 Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168765 done Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168758 WEXITSTATUS 1 Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168758 done Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168757 WEXITSTATUS 1 Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168757 done Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168764 WEXITSTATUS 1 Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168764 done Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168761 WEXITSTATUS 1 Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168761 done Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168760 WEXITSTATUS 1 Nov 05 09:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168760 done Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168759 WEXITSTATUS 1 Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168759 done Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168756 WEXITSTATUS 1 Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168756 done Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168762 WEXITSTATUS 1 Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168762 done Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168733 WEXITSTATUS 1 Nov 05 09:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168733 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168738 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168738 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168732 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168732 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168736 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168736 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168731 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168731 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168737 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168737 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168767 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168767 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168735 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168735 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168742 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168742 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168734 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168734 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168774 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168774 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168780 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168780 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168741 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168741 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168790 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168790 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168775 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168775 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168770 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168770 done Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168789 WEXITSTATUS 1 Nov 05 09:49:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168789 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168768 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168768 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168769 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168769 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168785 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168785 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168771 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168771 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168779 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168779 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168747 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168747 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168773 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168773 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168772 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168772 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168744 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168744 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168750 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168750 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168745 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168745 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168776 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168776 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168740 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168740 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168755 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168755 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168746 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168746 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168778 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168778 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168766 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168766 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168751 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168751 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168777 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168777 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168786 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168786 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168748 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168748 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168753 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168753 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168749 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168749 done Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168787 WEXITSTATUS 1 Nov 05 09:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168787 done Nov 05 09:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168739 WEXITSTATUS 1 Nov 05 09:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168739 done Nov 05 09:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168782 WEXITSTATUS 1 Nov 05 09:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168782 done Nov 05 09:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168752 WEXITSTATUS 1 Nov 05 09:49:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168752 done Nov 05 09:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168743 WEXITSTATUS 1 Nov 05 09:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168743 done Nov 05 09:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168781 WEXITSTATUS 1 Nov 05 09:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168781 done Nov 05 09:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168788 WEXITSTATUS 1 Nov 05 09:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168788 done Nov 05 09:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168754 WEXITSTATUS 1 Nov 05 09:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168754 done Nov 05 09:49:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168784 WEXITSTATUS 1 Nov 05 09:49:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168784 done Nov 05 09:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168783 WEXITSTATUS 1 Nov 05 09:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168783 done Nov 05 09:49:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158249 Nov 05 09:49:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7560(8166775) WEXITSTATUS 1 Nov 05 09:49:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7560(8166775) done Nov 05 09:49:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159209 WEXITSTATUS 0 Nov 05 09:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159209 done Nov 05 09:49:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154683 WEXITSTATUS 0 Nov 05 09:49:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154683 done Nov 05 09:49:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1952(8163535) WEXITSTATUS 0 Nov 05 09:49:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1952(8163535) done Nov 05 09:49:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:49:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168876 InitPrio=31906 usec=4911 Nov 05 09:49:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:50:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168793 uid 25715 Nov 05 09:50:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:50:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7751(8168854) WEXITSTATUS 1 Nov 05 09:50:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7751(8168854) done Nov 05 09:50:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:50:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168677 uid 21920 Nov 05 09:50:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168678 uid 21920 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:50:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:50:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168679 uid 21920 Nov 05 09:50:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168671 uid 21920 Nov 05 09:50:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168672 uid 21920 Nov 05 09:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3922(8167368) WEXITSTATUS 0 Nov 05 09:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3922(8167368) done Nov 05 09:50:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168673 uid 21920 Nov 05 09:50:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:50:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168674 uid 21920 Nov 05 09:50:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168675 uid 21920 Nov 05 09:50:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168676 uid 21920 Nov 05 09:50:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1960(8163592) WEXITSTATUS 0 Nov 05 09:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1960(8163592) done Nov 05 09:50:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168877 InitPrio=20039 usec=3723 Nov 05 09:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167262 WEXITSTATUS 0 Nov 05 09:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167262 done Nov 05 09:51:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5344(8159022) WEXITSTATUS 1 Nov 05 09:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5344(8159022) done Nov 05 09:51:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_31(8165387) WEXITSTATUS 0 Nov 05 09:51:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_31(8165387) done Nov 05 09:51:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6935(8165218) WEXITSTATUS 1 Nov 05 09:51:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6935(8165218) done Nov 05 09:51:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1940(8163400) WEXITSTATUS 0 Nov 05 09:51:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1940(8163400) done Nov 05 09:51:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7392(8166148) WEXITSTATUS 1 Nov 05 09:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7392(8166148) done Nov 05 09:51:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6722(8164833) WEXITSTATUS 1 Nov 05 09:51:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6722(8164833) done Nov 05 09:51:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158125_7(8158265) Nov 05 09:51:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168700 uid 21920 Nov 05 09:51:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168701 uid 21920 Nov 05 09:51:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168702 uid 21920 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_46(8168878) in pi_ohern on r917u29n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_47(8168879) in pi_ohern on r917u29n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_48(8168880) in pi_ohern on r917u29n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_49(8168881) in pi_ohern on r917u29n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_50(8168882) in pi_ohern on r917u29n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_51(8168883) in pi_ohern on r911u32n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_52(8168884) in pi_ohern on r911u32n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_53(8168885) in pi_ohern on r911u32n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_54(8168886) in pi_ohern on r911u32n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_55(8168887) in pi_ohern on r911u32n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_56(8168888) in pi_ohern on r911u32n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_57(8168889) in pi_ohern on r911u32n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_58(8168890) in pi_ohern on r911u32n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_59(8168891) in pi_ohern on r903u24n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_60(8168892) in pi_ohern on r903u24n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_61(8168893) in pi_ohern on r903u24n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_62(8168894) in pi_ohern on r903u24n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_63(8168895) in pi_ohern on r903u24n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_64(8168896) in pi_ohern on r903u24n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_65(8168897) in pi_ohern on r903u26n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_66(8168898) in pi_ohern on r903u26n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_67(8168899) in pi_ohern on r903u26n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_68(8168900) in pi_ohern on r903u26n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_69(8168901) in pi_ohern on r903u26n01 Nov 05 09:51:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_70(8168902) in pi_ohern on r903u26n01 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168877 in day on r815u03n[02-03,08],r815u09n[01,04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01,r816u15n03 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3940(8168903) in gpu on r903u34n01 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3941(8168904) in gpu on r808u08n01 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3942(8168905) in gpu on r808u08n01 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3943(8168906) in gpu on r808u14n01 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168561 in day on r909u01n01 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168703 uid 21920 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_87(7572254) WEXITSTATUS 0 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_87(7572254) done Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1091(7572057) WEXITSTATUS 0 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1091(7572057) done Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1108(7572074) WEXITSTATUS 0 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1108(7572074) done Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_396(7571361) WEXITSTATUS 0 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_396(7571361) done Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1168(7572135) WEXITSTATUS 0 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1168(7572135) done Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1157(7572124) WEXITSTATUS 0 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1157(7572124) done Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_866(7573035) WEXITSTATUS 0 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_866(7573035) done Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_945(7573114) WEXITSTATUS 0 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_945(7573114) done Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6676680_6(6676680) in gpu on r808u17n01 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2857(8168907) in day on r807u08n01 Nov 05 09:51:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2858(8168908) in day on r918u01n01 Nov 05 09:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_557(8116030) WEXITSTATUS 0 Nov 05 09:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_557(8116030) done Nov 05 09:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7568(8166796) WEXITSTATUS 1 Nov 05 09:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7568(8166796) done Nov 05 09:51:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168704 uid 21920 Nov 05 09:51:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7012(8165417) WEXITSTATUS 1 Nov 05 09:51:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7012(8165417) done Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168705 uid 21920 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5832(8162799) WEXITSTATUS 1 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5832(8162799) done Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7755(8168909) in scavenge on r907u05n01 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7756(8168910) in scavenge on r815u29n11 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7757(8168911) in scavenge on r816u29n09 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7758(8168912) in scavenge on r906u41n02 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7759(8168913) in scavenge on r909u10n04 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7760(8168914) in scavenge on r909u10n04 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7761(8168915) in scavenge on r909u10n04 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7762(8168916) in scavenge on r918u01n02 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7763(8168917) in scavenge on r918u09n04 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_699(7572868) in scavenge on r808u17n01 Nov 05 09:51:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1024(7573193) in scavenge on r815u23n05 Nov 05 09:51:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168706 uid 21920 Nov 05 09:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3942(8168905) WEXITSTATUS 0 Nov 05 09:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3942(8168905) done Nov 05 09:51:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168707 uid 21920 Nov 05 09:51:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168918 InitPrio=57385 usec=5332 Nov 05 09:51:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168918 NodeList=r915u38n03 #CPUs=1 Partition=pi_lora Nov 05 09:51:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:51:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168708 uid 21920 Nov 05 09:52:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3018(8129023) Nov 05 09:52:02 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3026(8129031) Nov 05 09:52:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168709 uid 21920 Nov 05 09:52:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168710 uid 21920 Nov 05 09:52:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168711 uid 21920 Nov 05 09:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3943(8168906) WEXITSTATUS 0 Nov 05 09:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3943(8168906) done Nov 05 09:52:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168712 uid 21920 Nov 05 09:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1979(8163838) WEXITSTATUS 0 Nov 05 09:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1979(8163838) done Nov 05 09:52:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168713 uid 21920 Nov 05 09:52:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168919 InitPrio=35865 usec=6878 Nov 05 09:52:15 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168919 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 09:52:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168920 InitPrio=57385 usec=3864 Nov 05 09:52:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168920 NodeList=r915u38n03 #CPUs=1 Partition=pi_lora Nov 05 09:52:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7736(8168727) WEXITSTATUS 1 Nov 05 09:52:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7736(8168727) done Nov 05 09:52:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:52:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:52:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7463(8166317) WEXITSTATUS 1 Nov 05 09:52:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7463(8166317) done Nov 05 09:52:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4301(8154067) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:52:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2376(8165583) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:52:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2377(8165584) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:52:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7582(8166820) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:52:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4301(8154067) Nov 05 09:52:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7582(8166820) Nov 05 09:52:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2376(8165583) Nov 05 09:52:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2377(8165584) Nov 05 09:52:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168921 InitPrio=57385 usec=3935 Nov 05 09:52:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168921 NodeList=r915u38n03 #CPUs=1 Partition=pi_lora Nov 05 09:52:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168562 uid 21920 Nov 05 09:52:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168563 uid 21920 Nov 05 09:52:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168564 uid 21920 Nov 05 09:52:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:52:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168565 uid 21920 Nov 05 09:52:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168566 uid 21920 Nov 05 09:52:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168567 uid 21920 Nov 05 09:52:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168922 InitPrio=28349 usec=4177 Nov 05 09:52:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=81685678 uid 21920 Nov 05 09:52:57 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=81685678 Nov 05 09:52:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=81685678 sig=9 returned: Invalid job id specified Nov 05 09:52:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168923 InitPrio=57385 usec=3644 Nov 05 09:53:01 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168923 NodeList=r915u38n03 #CPUs=1 Partition=pi_lora Nov 05 09:53:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168568 uid 21920 Nov 05 09:53:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:53:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168569 uid 21920 Nov 05 09:53:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7037(8165471) WEXITSTATUS 1 Nov 05 09:53:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7037(8165471) done Nov 05 09:53:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168571 uid 21920 Nov 05 09:53:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7462(8166316) WEXITSTATUS 1 Nov 05 09:53:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7462(8166316) done Nov 05 09:53:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168572 uid 21920 Nov 05 09:53:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168570 uid 21920 Nov 05 09:53:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168573 uid 21920 Nov 05 09:53:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168574 uid 21920 Nov 05 09:53:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168575 uid 21920 Nov 05 09:53:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168924 InitPrio=57385 usec=3680 Nov 05 09:53:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168924 NodeList=r915u38n03 #CPUs=1 Partition=pi_lora Nov 05 09:53:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168576 uid 21920 Nov 05 09:53:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168577 uid 21920 Nov 05 09:53:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168578 uid 21920 Nov 05 09:53:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168579 uid 21920 Nov 05 09:53:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158125_9(8158267) Nov 05 09:53:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7236(8165889) WEXITSTATUS 1 Nov 05 09:53:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7236(8165889) done Nov 05 09:53:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168925 InitPrio=57385 usec=3801 Nov 05 09:53:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168925 NodeList=r915u38n03 #CPUs=1 Partition=pi_lora Nov 05 09:53:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2614(8152881) WEXITSTATUS 0 Nov 05 09:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2614(8152881) done Nov 05 09:53:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7029(8165454) WEXITSTATUS 1 Nov 05 09:53:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7029(8165454) done Nov 05 09:53:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7459(8166313) WEXITSTATUS 1 Nov 05 09:53:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7459(8166313) done Nov 05 09:53:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168926 InitPrio=57385 usec=3694 Nov 05 09:53:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1004(7573173) WEXITSTATUS 0 Nov 05 09:53:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1004(7573173) done Nov 05 09:53:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168580 uid 21920 Nov 05 09:53:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168926 NodeList=r801u30n01 #CPUs=1 Partition=pi_lora Nov 05 09:53:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:53:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168581 uid 21920 Nov 05 09:54:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168582 uid 21920 Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_0(7829890) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_1(7829891) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_2(7829892) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_3(7829893) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_4(7829894) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_5(7829895) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_6(7829896) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_7(7829897) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_8(7829898) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_9(7829899) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_10(7829900) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_11(7829901) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_12(7829902) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_13(7829903) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_14(7829904) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_15(7829905) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_16(7829906) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_17(7829907) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_18(7829908) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_19(7829909) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_20(7829910) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_21(7829911) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_22(7829912) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_23(7829913) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_24(7829914) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_25(7829915) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_26(7829916) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_27(7829917) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_28(7829918) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_29(7829919) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_30(7829920) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_31(7829921) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_32(7829922) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_33(7829923) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_34(7829924) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_35(7829925) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_36(7829926) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_37(7829927) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_38(7829928) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_39(7829929) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_40(7829930) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_41(7829931) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_42(7829932) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_43(7829933) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_44(7829934) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_45(7829935) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_46(7829936) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_47(7829937) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_48(7829938) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_49(7829939) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_50(7829940) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_51(7829941) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_52(7829942) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_53(7829943) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_54(7829944) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_55(7829945) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_56(7829946) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_57(7829947) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_58(7829948) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_59(7829949) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_60(7829950) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_61(7829951) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_62(7829952) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_63(7829953) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_64(7829954) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_65(7829955) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_66(7829956) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_67(7829957) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_68(7829958) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_69(7829959) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_70(7829960) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_71(7829961) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_72(7829962) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_73(7829963) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_74(7829964) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_75(7829965) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_76(7829966) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_77(7829967) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_78(7829968) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_79(7829969) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_80(7829970) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_81(7829971) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_82(7829972) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_83(7829973) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_84(7829974) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_85(7829975) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_86(7829976) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_87(7829977) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_88(7829978) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_89(7829979) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_90(7829980) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_91(7829981) Nov 05 09:54:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7829889_92(7829982) Nov 05 09:54:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168583 uid 21920 Nov 05 09:54:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168584 uid 21920 Nov 05 09:54:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168585 uid 21920 Nov 05 09:54:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8168927 InitPrio=57385 usec=3676 Nov 05 09:54:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168927 NodeList=r801u30n01 #CPUs=1 Partition=pi_lora Nov 05 09:54:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168586 uid 21920 Nov 05 09:54:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168587 uid 21920 Nov 05 09:54:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168588 uid 21920 Nov 05 09:54:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168589 uid 21920 Nov 05 09:54:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168590 uid 21920 Nov 05 09:54:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168591 uid 21920 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_71(8168928) in pi_ohern on r801u32n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_72(8168929) in pi_ohern on r801u32n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_73(8168930) in pi_ohern on r801u32n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_74(8168931) in pi_ohern on r801u32n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_75(8168932) in pi_ohern on r801u32n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_76(8168933) in pi_ohern on r801u32n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_77(8168934) in pi_ohern on r801u32n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_78(8168935) in pi_ohern on r801u32n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_79(8168936) in pi_ohern on r917u13n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_80(8168937) in pi_ohern on r917u13n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_81(8168938) in pi_ohern on r917u13n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_82(8168939) in pi_ohern on r917u13n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_83(8168940) in pi_ohern on r917u13n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_84(8168941) in pi_ohern on r917u13n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_85(8168942) in pi_ohern on r917u13n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_86(8168943) in pi_ohern on r917u13n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_87(8168944) in pi_ohern on r917u13n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_88(8168945) in pi_ohern on r917u13n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_89(8168946) in pi_ohern on r917u13n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_90(8168947) in pi_ohern on r917u13n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_91(8168948) in pi_ohern on r917u13n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_92(8168949) in pi_ohern on r917u29n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_93(8168950) in pi_ohern on r917u29n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_94(8168951) in pi_ohern on r917u29n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_95(8168952) in pi_ohern on r917u29n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_96(8168953) in pi_ohern on r917u29n02 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_97(8168954) in pi_ohern on r917u29n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_98(8168955) in pi_ohern on r917u29n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_99(8168956) in pi_ohern on r917u29n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_100(8168957) in pi_ohern on r917u29n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_101(8168958) in pi_ohern on r917u29n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_102(8168959) in pi_ohern on r917u29n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_103(8168960) in pi_ohern on r917u29n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_104(8168961) in pi_ohern on r917u29n03 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_105(8168962) in pi_ohern on r917u29n04 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_106(8168963) in pi_ohern on r917u29n04 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_107(8168964) in pi_ohern on r917u29n04 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_108(8168965) in pi_ohern on r917u29n04 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_109(8168966) in pi_ohern on r917u29n04 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_110(8168967) in pi_ohern on r913u35n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_111(8168968) in pi_ohern on r913u35n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_112(8168969) in pi_ohern on r913u35n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_113(8168970) in pi_ohern on r913u35n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_114(8168971) in pi_ohern on r913u35n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_115(8168972) in pi_ohern on r913u35n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_116(8168973) in pi_ohern on r913u35n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_117(8168974) in pi_ohern on r913u35n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_118(8168975) in pi_ohern on r903u12n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_119(8168976) in pi_ohern on r903u12n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_120(8168977) in pi_ohern on r903u12n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_121(8168978) in pi_ohern on r903u12n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_122(8168979) in pi_ohern on r903u12n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_123(8168980) in pi_ohern on r903u12n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_124(8168981) in pi_ohern on r903u12n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_125(8168982) in pi_ohern on r903u12n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_126(8168983) in pi_ohern on r903u24n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_127(8168984) in pi_ohern on r903u24n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_128(8168985) in pi_ohern on r903u24n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_129(8168986) in pi_ohern on r903u26n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_130(8168987) in pi_ohern on r903u26n01 Nov 05 09:54:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_131(8168988) in pi_ohern on r903u26n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_109(8168989) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_110(8168990) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_111(8168991) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_112(8168992) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_113(8168993) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_114(8168994) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_115(8168995) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_116(8168996) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_117(8168997) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_153(7572321) WEXITSTATUS 0 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_153(7572321) done Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_188(7572356) WEXITSTATUS 0 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_188(7572356) done Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1034(7573203) WEXITSTATUS 0 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1034(7573203) done Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_955(7573124) WEXITSTATUS 0 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_955(7573124) done Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_63(7572230) WEXITSTATUS 0 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_63(7572230) done Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_57(8114323) WEXITSTATUS 0 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_57(8114323) done Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168592 uid 21920 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_118(8168998) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_119(8168999) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_120(8169000) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_121(8169001) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_122(8169002) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_123(8169003) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_124(8169004) in week on r816u23n01 Nov 05 09:54:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_125(8169005) in week on r816u23n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_126(8169006) in week on r816u23n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_127(8169007) in week on r816u23n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_128(8169008) in week on r816u23n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_129(8169009) in week on r816u23n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_130(8169010) in week on r816u23n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_131(8169011) in week on r816u23n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_132(8169012) in week on r816u23n03 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_133(8169013) in week on r816u23n03 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_134(8169014) in week on r816u23n04 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_135(8169015) in week on r816u23n04 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_136(8169016) in week on r816u23n04 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_137(8169017) in week on r816u23n04 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_138(8169018) in week on r816u23n06 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_139(8169019) in week on r816u23n06 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_140(8169020) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_141(8169021) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_142(8169022) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_143(8169023) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_144(8169024) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_145(8169025) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_146(8169026) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_147(8169027) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_148(8169028) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_149(8169029) in week on r816u29n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_150(8169030) in week on r816u29n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_151(8169031) in week on r816u29n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_152(8169032) in week on r816u29n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_153(8169033) in week on r816u29n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7829889_154(7829889) in week on r816u29n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168922 in gpu on r808u08n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3944(8169034) in gpu on r808u14n01 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168538 in day on r805u23n02 Nov 05 09:54:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2859(8169035) in day on r919u32n01 Nov 05 09:54:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168593 uid 21920 Nov 05 09:54:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169036 InitPrio=57385 usec=2180 Nov 05 09:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_483(7572651) WEXITSTATUS 0 Nov 05 09:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_483(7572651) done Nov 05 09:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_699(7572868) WEXITSTATUS 0 Nov 05 09:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_699(7572868) done Nov 05 09:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133393 WEXITSTATUS 0 Nov 05 09:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133393 done Nov 05 09:54:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168594 uid 21920 Nov 05 09:54:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169036 NodeList=r801u30n01 #CPUs=1 Partition=pi_lora Nov 05 09:54:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7764(8169037) in scavenge on r801u06n03 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7765(8169038) in scavenge on r903u07n01 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7766(8169039) in scavenge on r815u09n10 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7767(8169040) in scavenge on r906u28n01 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7768(8169041) in scavenge on r805u09n01 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7769(8169042) in scavenge on r806u06n01 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7770(8169043) in scavenge on r807u11n02 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7771(8169044) in scavenge on r906u41n03 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7772(8169045) in scavenge on r909u04n04 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7773(8169046) in scavenge on r909u13n01 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7774(8169047) in scavenge on r919u01n04 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1062(7573231) in scavenge on r816u29n02 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1110(7573279) in scavenge on r816u29n03 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1183(7573352) in scavenge on r816u29n04 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_143(8114409) in scavenge on r816u29n06 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_206(8114472) in scavenge on r816u35n01 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_266(8114532) in scavenge on r816u35n01 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_330(8114597) in scavenge on r816u35n01 Nov 05 09:54:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_657(8114926) in scavenge on r816u35n03 Nov 05 09:54:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168595 uid 21920 Nov 05 09:54:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_840(7573009) Nov 05 09:54:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3035(8129145) Nov 05 09:54:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168596 uid 21920 Nov 05 09:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1977(8163836) WEXITSTATUS 0 Nov 05 09:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1977(8163836) done Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:54:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168597 uid 21920 Nov 05 09:54:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168598 uid 21920 Nov 05 09:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1963(8163639) WEXITSTATUS 0 Nov 05 09:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1963(8163639) done Nov 05 09:54:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168599 uid 21920 Nov 05 09:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168600 uid 21920 Nov 05 09:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7480(8166437) WEXITSTATUS 1 Nov 05 09:54:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7480(8166437) done Nov 05 09:54:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:54:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_92(7572259) WEXITSTATUS 0 Nov 05 09:54:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_92(7572259) done Nov 05 09:54:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3938(8168715) WEXITSTATUS 0 Nov 05 09:54:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3938(8168715) done Nov 05 09:54:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:01 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7337(8166041) has been requeued to reclaim resources for JobId=Invalid Nov 05 09:55:03 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7337(8166041) Nov 05 09:55:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6378(8164049) WEXITSTATUS 1 Nov 05 09:55:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6378(8164049) done Nov 05 09:55:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7241(8165898) WEXITSTATUS 1 Nov 05 09:55:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7241(8165898) done Nov 05 09:55:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169048 InitPrio=21331 usec=6215 Nov 05 09:55:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7639(8167094) WEXITSTATUS 1 Nov 05 09:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7639(8167094) done Nov 05 09:55:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6389(8164100) WEXITSTATUS 1 Nov 05 09:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6389(8164100) done Nov 05 09:55:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3940(8168903) WEXITSTATUS 0 Nov 05 09:55:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3940(8168903) done Nov 05 09:55:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168876 NodeList=r816u35n07,r907u32n01 #CPUs=8 Partition=gpu Nov 05 09:55:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169048 NodeList=r903u37n01 #CPUs=1 Partition=gpu_devel Nov 05 09:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3944(8169034) WEXITSTATUS 0 Nov 05 09:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3944(8169034) done Nov 05 09:55:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:55:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168601 uid 21920 Nov 05 09:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168876 WEXITSTATUS 1 Nov 05 09:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168876 done Nov 05 09:56:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1970(8163741) WEXITSTATUS 0 Nov 05 09:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1970(8163741) done Nov 05 09:56:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168602 uid 21920 Nov 05 09:56:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154709 WEXITSTATUS 0 Nov 05 09:56:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154709 done Nov 05 09:56:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168603 uid 21920 Nov 05 09:56:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168604 uid 21920 Nov 05 09:56:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168605 uid 21920 Nov 05 09:56:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154696 WEXITSTATUS 0 Nov 05 09:56:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154696 done Nov 05 09:56:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168606 uid 21920 Nov 05 09:56:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168607 uid 21920 Nov 05 09:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1530(8145436) WEXITSTATUS 0 Nov 05 09:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1530(8145436) done Nov 05 09:56:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168608 uid 21920 Nov 05 09:56:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168536 WEXITSTATUS 0 Nov 05 09:56:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168536 done Nov 05 09:56:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168609 uid 21920 Nov 05 09:56:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168610 uid 21920 Nov 05 09:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2593(8152860) WEXITSTATUS 0 Nov 05 09:56:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2593(8152860) done Nov 05 09:56:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168611 uid 21920 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:56:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168612 uid 21920 Nov 05 09:56:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168613 uid 21920 Nov 05 09:56:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1933(8163335) WEXITSTATUS 0 Nov 05 09:56:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1933(8163335) done Nov 05 09:56:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168614 uid 21920 Nov 05 09:56:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168615 uid 21920 Nov 05 09:56:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168616 uid 21920 Nov 05 09:56:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168617 uid 21920 Nov 05 09:56:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166476 WEXITSTATUS 0 Nov 05 09:56:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166476 done Nov 05 09:56:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168618 uid 21920 Nov 05 09:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_302(8114568) WEXITSTATUS 0 Nov 05 09:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_302(8114568) done Nov 05 09:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_249(7572417) WEXITSTATUS 0 Nov 05 09:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_249(7572417) done Nov 05 09:56:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168455 NodeList=r816u15n08 #CPUs=16 Partition=day Nov 05 09:56:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168619 uid 21920 Nov 05 09:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3873(8166038) WEXITSTATUS 0 Nov 05 09:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3873(8166038) done Nov 05 09:56:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:56:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168620 uid 21920 Nov 05 09:56:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168621 uid 21920 Nov 05 09:56:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168622 uid 21920 Nov 05 09:56:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168623 uid 21920 Nov 05 09:57:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168624 uid 21920 Nov 05 09:57:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168625 uid 21920 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_132(8169049) in pi_ohern on r801u32n02 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_133(8169050) in pi_ohern on r801u32n02 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_134(8169051) in pi_ohern on r801u32n02 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_135(8169052) in pi_ohern on r801u32n02 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_136(8169053) in pi_ohern on r801u32n02 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_137(8169054) in pi_ohern on r801u32n02 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_138(8169055) in pi_ohern on r801u32n02 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_139(8169056) in pi_ohern on r801u32n02 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_140(8169057) in pi_ohern on r917u13n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_141(8169058) in pi_ohern on r917u13n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_142(8169059) in pi_ohern on r917u13n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_143(8169060) in pi_ohern on r917u13n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_144(8169061) in pi_ohern on r917u13n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_145(8169062) in pi_ohern on r917u13n04 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_146(8169063) in pi_ohern on r917u13n04 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_147(8169064) in pi_ohern on r917u29n04 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_148(8169065) in pi_ohern on r917u29n04 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_149(8169066) in pi_ohern on r917u29n04 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_150(8169067) in pi_ohern on r917u29n04 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_151(8169068) in pi_ohern on r917u29n04 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_152(8169069) in pi_ohern on r917u29n04 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_153(8169070) in pi_ohern on r807u22n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_154(8169071) in pi_ohern on r807u22n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_155(8169072) in pi_ohern on r807u22n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_156(8169073) in pi_ohern on r807u22n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_157(8169074) in pi_ohern on r807u22n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_158(8169075) in pi_ohern on r807u22n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_159(8169076) in pi_ohern on r807u22n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_160(8169077) in pi_ohern on r807u22n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_161(8169078) in pi_ohern on r807u24n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_162(8169079) in pi_ohern on r807u24n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_163(8169080) in pi_ohern on r807u24n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_164(8169081) in pi_ohern on r807u24n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_165(8169082) in pi_ohern on r807u24n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_166(8169083) in pi_ohern on r807u24n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_167(8169084) in pi_ohern on r903u12n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_168(8169085) in pi_ohern on r903u12n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_169(8169086) in pi_ohern on r903u12n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_170(8169087) in pi_ohern on r903u12n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_171(8169088) in pi_ohern on r903u12n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_172(8169089) in pi_ohern on r903u12n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_173(8169090) in pi_ohern on r903u12n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_174(8169091) in pi_ohern on r903u12n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_175(8169092) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_176(8169093) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_177(8169094) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_178(8169095) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_179(8169096) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_180(8169097) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_181(8169098) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_182(8169099) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_183(8169100) in pi_ohern on r903u26n01 Nov 05 09:57:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_184(8169101) in pi_ohern on r903u26n01 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168626 uid 21920 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_393(7572561) WEXITSTATUS 0 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_393(7572561) done Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_340(7572508) WEXITSTATUS 0 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_340(7572508) done Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_708(7572877) WEXITSTATUS 0 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_708(7572877) done Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_112(8114378) WEXITSTATUS 0 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_112(8114378) done Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_549(7572718) WEXITSTATUS 0 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_549(7572718) done Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_805(7572974) WEXITSTATUS 0 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_805(7572974) done Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_865(8115134) WEXITSTATUS 0 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_865(8115134) done Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_584(7572753) WEXITSTATUS 0 Nov 05 09:57:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_584(7572753) done Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3945(8169102) in gpu on r903u34n01 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3946(8169103) in gpu on r808u14n01 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3947(8169104) in gpu on r808u14n01 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3948(8169105) in gpu on r808u17n01 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3949(8169106) in gpu on r4513u29n01 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168627 in day on r805u02n04 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168628 in day on r805u25n04 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168539 in day on r919u32n04 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u05n03 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2860(8169107) in day on r805u27n01 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2861(8169108) in day on r914u05n01 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2862(8169109) in day on r918u01n03 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2863(8169110) in day on r918u05n01 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2864(8169111) in day on r918u36n04 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2865(8169112) in day on r815u09n03 Nov 05 09:57:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2866(8169113) in day on r815u09n03 Nov 05 09:57:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1014(7573183) WEXITSTATUS 0 Nov 05 09:57:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1014(7573183) done Nov 05 09:57:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168627 uid 21920 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7775(8169114) in scavenge on r805u04n02 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7776(8169115) in scavenge on r806u04n04 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7777(8169116) in scavenge on r913u28n01 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7778(8169117) in scavenge on r914u05n03 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7779(8169118) in scavenge on r918u36n04 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7780(8169119) in scavenge on r918u36n04 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7781(8169120) in scavenge on r919u09n02 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_156(8115626) in scavenge on r907u32n01 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_646(8116119) in scavenge on r907u34n01 Nov 05 09:57:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1077(7572043) in scavenge on r816u35n02 Nov 05 09:57:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2376(8165583) in scavenge on r907u32n01 Nov 05 09:57:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2377(8165584) in scavenge on r907u34n01 Nov 05 09:57:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168628 uid 21920 Nov 05 09:57:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168629 uid 21920 Nov 05 09:57:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168630 uid 21920 Nov 05 09:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3945(8169102) WEXITSTATUS 0 Nov 05 09:57:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3945(8169102) done Nov 05 09:57:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:57:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168631 uid 21920 Nov 05 09:57:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168632 uid 21920 Nov 05 09:57:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168633 uid 21920 Nov 05 09:57:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168634 uid 21920 Nov 05 09:57:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168635 uid 21920 Nov 05 09:57:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168636 uid 21920 Nov 05 09:57:32 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8169121 NodeList=(null) usec=26480 Nov 05 09:57:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168637 uid 21920 Nov 05 09:57:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169121 NodeList=r802u27n01 #CPUs=10 Partition=admintest Nov 05 09:57:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:57:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168638 uid 21920 Nov 05 09:57:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168639 uid 21920 Nov 05 09:57:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168640 uid 21920 Nov 05 09:57:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169122 InitPrio=31906 usec=3696 Nov 05 09:57:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:57:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_156(8115626) WEXITSTATUS 0 Nov 05 09:57:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_156(8115626) done Nov 05 09:57:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168641 uid 21920 Nov 05 09:57:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:57:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169122 NodeList=r816u35n07,r907u32n01 #CPUs=8 Partition=gpu Nov 05 09:57:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168642 uid 21920 Nov 05 09:57:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168643 uid 21920 Nov 05 09:57:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168644 uid 21920 Nov 05 09:57:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:57:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168645 uid 21920 Nov 05 09:57:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168646 uid 21920 Nov 05 09:57:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169122 WEXITSTATUS 1 Nov 05 09:57:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169122 done Nov 05 09:57:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:57:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168647 uid 21920 Nov 05 09:57:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168648 uid 21920 Nov 05 09:58:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168649 uid 21920 Nov 05 09:58:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168650 uid 21920 Nov 05 09:58:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168651 uid 21920 Nov 05 09:58:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168652 uid 21920 Nov 05 09:58:05 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8168652 Nov 05 09:58:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=8168652 sig=9 returned: Invalid job id specified Nov 05 09:58:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168653 uid 21920 Nov 05 09:58:07 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8168653 Nov 05 09:58:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=8168653 sig=9 returned: Invalid job id specified Nov 05 09:58:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168654 uid 21920 Nov 05 09:58:09 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8168654 Nov 05 09:58:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=8168654 sig=9 returned: Invalid job id specified Nov 05 09:58:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168655 uid 21920 Nov 05 09:58:11 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8168655 Nov 05 09:58:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=8168655 sig=9 returned: Invalid job id specified Nov 05 09:58:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168656 uid 21920 Nov 05 09:58:13 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8168656 Nov 05 09:58:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=8168656 sig=9 returned: Invalid job id specified Nov 05 09:58:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168657 uid 21920 Nov 05 09:58:15 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8168657 Nov 05 09:58:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=8168657 sig=9 returned: Invalid job id specified Nov 05 09:58:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6629(8164694) WEXITSTATUS 1 Nov 05 09:58:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6629(8164694) done Nov 05 09:58:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:58:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168658 uid 21920 Nov 05 09:58:17 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8168658 Nov 05 09:58:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=8168658 sig=9 returned: Invalid job id specified Nov 05 09:58:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168659 uid 21920 Nov 05 09:58:19 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=8168659 Nov 05 09:58:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=21920 JobId=8168659 sig=9 returned: Invalid job id specified Nov 05 09:58:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168660 uid 21920 Nov 05 09:58:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168661 uid 21920 Nov 05 09:58:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168922 uid 25715 Nov 05 09:58:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:58:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168662 uid 21920 Nov 05 09:58:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168663 uid 21920 Nov 05 09:58:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168664 uid 21920 Nov 05 09:58:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168665 uid 21920 Nov 05 09:58:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 09:58:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:58:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168666 uid 21920 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 09:58:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 09:58:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168667 uid 21920 Nov 05 09:58:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3949(8169106) WEXITSTATUS 0 Nov 05 09:58:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3949(8169106) done Nov 05 09:58:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168668 uid 21920 Nov 05 09:58:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:58:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168669 uid 21920 Nov 05 09:58:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168670 uid 21920 Nov 05 09:58:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7062(8165525) WEXITSTATUS 1 Nov 05 09:58:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7062(8165525) done Nov 05 09:58:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133377 WEXITSTATUS 0 Nov 05 09:58:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133377 done Nov 05 09:58:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7731(8168722) WEXITSTATUS 1 Nov 05 09:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7731(8168722) done Nov 05 09:58:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:58:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8169123 NodeList=(null) usec=57288 Nov 05 09:58:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169123 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 09:58:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:58:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154659 WEXITSTATUS 0 Nov 05 09:58:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154659 done Nov 05 09:59:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907441 WEXITSTATUS 0 Nov 05 09:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907441 done Nov 05 09:59:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169124 InitPrio=25838 usec=3640 Nov 05 09:59:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7520(8166539) WEXITSTATUS 1 Nov 05 09:59:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7520(8166539) done Nov 05 09:59:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169125 InitPrio=63372 usec=3914 Nov 05 09:59:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6737(8164858) WEXITSTATUS 1 Nov 05 09:59:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6737(8164858) done Nov 05 09:59:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:31 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_921(7573090) Nov 05 09:59:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169126 InitPrio=10612 usec=2057 Nov 05 09:59:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169127 InitPrio=10612 usec=3213 Nov 05 09:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168291 WEXITSTATUS 0 Nov 05 09:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168291 done Nov 05 09:59:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169128 InitPrio=57391 usec=4037 Nov 05 09:59:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144145 WEXITSTATUS 0 Nov 05 09:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144145 done Nov 05 09:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144143 WEXITSTATUS 0 Nov 05 09:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144143 done Nov 05 09:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144157 WEXITSTATUS 0 Nov 05 09:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144157 done Nov 05 09:59:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169129 InitPrio=10612 usec=2201 Nov 05 09:59:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169130 InitPrio=10612 usec=3679 Nov 05 09:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168295 WEXITSTATUS 0 Nov 05 09:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168295 done Nov 05 09:59:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:41 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8169131 NodeList=(null) usec=24967 Nov 05 09:59:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_185(8169132) in pi_ohern on r801u32n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_186(8169133) in pi_ohern on r917u13n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_187(8169134) in pi_ohern on r917u13n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_188(8169135) in pi_ohern on r917u13n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_189(8169136) in pi_ohern on r917u13n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_190(8169137) in pi_ohern on r917u13n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_191(8169138) in pi_ohern on r917u13n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_192(8169139) in pi_ohern on r917u13n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_193(8169140) in pi_ohern on r917u13n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_194(8169141) in pi_ohern on r917u13n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_195(8169142) in pi_ohern on r917u13n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_196(8169143) in pi_ohern on r917u13n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_197(8169144) in pi_ohern on r917u13n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_198(8169145) in pi_ohern on r917u13n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_199(8169146) in pi_ohern on r917u13n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_200(8169147) in pi_ohern on r917u13n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_201(8169148) in pi_ohern on r917u13n03 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_202(8169149) in pi_ohern on r917u13n03 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_203(8169150) in pi_ohern on r917u13n03 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_204(8169151) in pi_ohern on r917u13n03 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_205(8169152) in pi_ohern on r917u13n03 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_206(8169153) in pi_ohern on r917u13n03 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_207(8169154) in pi_ohern on r917u29n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_208(8169155) in pi_ohern on r917u29n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_209(8169156) in pi_ohern on r917u29n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_210(8169157) in pi_ohern on r917u29n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_211(8169158) in pi_ohern on r917u29n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_212(8169159) in pi_ohern on r917u29n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_213(8169160) in pi_ohern on r917u29n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_214(8169161) in pi_ohern on r917u29n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_215(8169162) in pi_ohern on r917u29n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_216(8169163) in pi_ohern on r917u29n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_217(8169164) in pi_ohern on r917u29n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_218(8169165) in pi_ohern on r917u29n02 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_219(8169166) in pi_ohern on r917u29n04 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_220(8169167) in pi_ohern on r917u29n04 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_221(8169168) in pi_ohern on r917u29n04 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_222(8169169) in pi_ohern on r917u29n04 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_223(8169170) in pi_ohern on r917u29n04 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_224(8169171) in pi_ohern on r917u29n04 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_225(8169172) in pi_ohern on r913u32n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_226(8169173) in pi_ohern on r913u32n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_227(8169174) in pi_ohern on r913u32n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_228(8169175) in pi_ohern on r913u32n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_229(8169176) in pi_ohern on r913u32n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_230(8169177) in pi_ohern on r913u32n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_231(8169178) in pi_ohern on r913u32n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_232(8169179) in pi_ohern on r913u32n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_233(8169180) in pi_ohern on r913u35n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_234(8169181) in pi_ohern on r913u35n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_235(8169182) in pi_ohern on r913u35n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_236(8169183) in pi_ohern on r913u35n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_237(8169184) in pi_ohern on r913u35n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_238(8169185) in pi_ohern on r913u35n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_239(8169186) in pi_ohern on r913u35n01 Nov 05 09:59:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_240(8169187) in pi_ohern on r913u35n01 Nov 05 09:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3950(8169188) in gpu on r903u34n01 Nov 05 09:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3951(8169189) in gpu on r808u08n01 Nov 05 09:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168680 in day on r805u02n04 Nov 05 09:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168681 in day on r805u25n04 Nov 05 09:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168684 in day on r816u09n07 Nov 05 09:59:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168680 uid 21920 Nov 05 09:59:46 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:132 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 09:59:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166583 in gpu on r4513u29n01 Nov 05 09:59:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166584 in gpu on r805u38n01 Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168681 uid 21920 Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_217(8114483) WEXITSTATUS 0 Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_217(8114483) done Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_666(7572835) WEXITSTATUS 0 Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_666(7572835) done Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_138(8114404) WEXITSTATUS 0 Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_138(8114404) done Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1162(7572129) WEXITSTATUS 0 Nov 05 09:59:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1162(7572129) done Nov 05 09:59:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168682 uid 21920 Nov 05 09:59:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169125 NodeList=r912u13n01 #CPUs=1 Partition=bigmem Nov 05 09:59:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144146 NodeList=r801u23n01,r917u10n02 #CPUs=84 Partition=scavenge Nov 05 09:59:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168474 WEXITSTATUS 0 Nov 05 09:59:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168474 done Nov 05 09:59:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168683 uid 21920 Nov 05 09:59:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7782(8169190) in scavenge on r802u09n02 Nov 05 09:59:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7783(8169191) in scavenge on r802u11n02 Nov 05 09:59:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7784(8169192) in scavenge on r805u11n03 Nov 05 09:59:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7785(8169193) in scavenge on r904u04n02 Nov 05 09:59:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7786(8169194) in scavenge on r919u36n03 Nov 05 09:59:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1196(7570858) in scavenge on r816u23n10 Nov 05 09:59:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1085(7572051) in scavenge on r907u32n01 Nov 05 09:59:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_377(7572545) in scavenge on r805u38n01 Nov 05 09:59:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_654(7572823) in scavenge on r806u37n01 Nov 05 09:59:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168684 uid 21920 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2867(8169195) in scavenge on r816u29n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2868(8169196) in scavenge on r816u29n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2869(8169197) in scavenge on r816u29n04 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2870(8169198) in scavenge on r816u29n04 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2871(8169199) in scavenge on r816u29n05 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2872(8169200) in scavenge on r816u29n05 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2873(8169201) in scavenge on r816u29n06 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2874(8169202) in scavenge on r816u29n06 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2875(8169203) in scavenge on r816u35n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2876(8169204) in scavenge on r816u35n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2877(8169205) in scavenge on r816u35n04 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2878(8169206) in scavenge on r805u34n01 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8089587 uid 22341 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_1(8169207) in scavenge on r801u30n01 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_2(8169208) in scavenge on r908u37n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_3(8169209) in scavenge on r907u01n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_4(8169210) in scavenge on r907u01n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_5(8169211) in scavenge on r907u01n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_6(8169212) in scavenge on r907u01n03 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_7(8169213) in scavenge on r907u01n03 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_8(8169214) in scavenge on r907u01n03 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_9(8169215) in scavenge on r907u01n04 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169126_10(8169126) in scavenge on r907u01n04 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_1(8169216) in scavenge on r907u01n04 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_2(8169217) in scavenge on r907u05n01 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_3(8169218) in scavenge on r907u05n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_4(8169219) in scavenge on r907u05n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_5(8169220) in scavenge on r907u05n02 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_6(8169221) in scavenge on r907u05n03 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_7(8169222) in scavenge on r907u05n03 Nov 05 09:59:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_8(8169223) in scavenge on r907u05n03 Nov 05 09:59:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_9(8169224) in scavenge on r907u05n04 Nov 05 09:59:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169129_10(8169129) in scavenge on r907u05n04 Nov 05 09:59:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168685 uid 21920 Nov 05 10:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168686 uid 21920 Nov 05 10:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 10:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 10:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 10:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 10:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 10:00:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168687 uid 21920 Nov 05 10:00:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168688 uid 21920 Nov 05 10:00:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168689 uid 21920 Nov 05 10:00:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168690 uid 21920 Nov 05 10:00:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168691 uid 21920 Nov 05 10:00:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168692 uid 21920 Nov 05 10:00:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1511(8143570) WEXITSTATUS 0 Nov 05 10:00:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1511(8143570) done Nov 05 10:00:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168693 uid 21920 Nov 05 10:00:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168694 uid 21920 Nov 05 10:00:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168695 uid 21920 Nov 05 10:00:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1945(8163405) WEXITSTATUS 0 Nov 05 10:00:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1945(8163405) done Nov 05 10:00:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168696 uid 21920 Nov 05 10:00:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168697 uid 21920 Nov 05 10:00:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168698 uid 21920 Nov 05 10:00:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168699 uid 21920 Nov 05 10:00:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8089587 error Job/step already completing or completed Nov 05 10:00:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8163239 has been requeued to reclaim resources for JobId=8169128 Nov 05 10:00:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6684(8164786) has been requeued to reclaim resources for JobId=8169128 Nov 05 10:00:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4178(8151237) has been requeued to reclaim resources for JobId=8169128 Nov 05 10:00:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8159213 has been requeued to reclaim resources for JobId=8169128 Nov 05 10:00:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8163214 has been requeued to reclaim resources for JobId=8169128 Nov 05 10:00:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8163238 has been requeued to reclaim resources for JobId=8169128 Nov 05 10:00:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169124 NodeList=r807u14n01 #CPUs=8 Partition=pi_panda Nov 05 10:00:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6684(8164786) Nov 05 10:00:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8163239 Nov 05 10:00:32 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8159213 Nov 05 10:00:32 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8163238 Nov 05 10:00:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4178(8151237) Nov 05 10:00:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7019(8165436) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:00:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8163214 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:00:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7019(8165436) Nov 05 10:00:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169128 NodeList=r801u30n04,r915u38n[01-02,04],r917u04n01 #CPUs=216 Partition=pi_lora Nov 05 10:00:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3948(8169105) WEXITSTATUS 0 Nov 05 10:00:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3948(8169105) done Nov 05 10:00:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7563(8166778) WEXITSTATUS 1 Nov 05 10:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7563(8166778) done Nov 05 10:00:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:00:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1934(8163360) WEXITSTATUS 0 Nov 05 10:00:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1934(8163360) done Nov 05 10:00:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7770(8169043) WEXITSTATUS 1 Nov 05 10:00:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7770(8169043) done Nov 05 10:00:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:00:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:00:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r904u07n03 #CPUs=1 Partition=day Nov 05 10:00:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r904u07n03 #CPUs=1 Partition=day Nov 05 10:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169131 WTERMSIG 2 Nov 05 10:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169131 cancelled by interactive user Nov 05 10:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169131 done Nov 05 10:01:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8169131 error Job/step already completing or completed Nov 05 10:01:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3933(8168492) WEXITSTATUS 0 Nov 05 10:01:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3933(8168492) done Nov 05 10:01:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8169225 NodeList=(null) usec=23663 Nov 05 10:01:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 10:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 10:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7643(8167111) WEXITSTATUS 1 Nov 05 10:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7643(8167111) done Nov 05 10:01:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 10:01:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154594 WEXITSTATUS 0 Nov 05 10:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154594 done Nov 05 10:01:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7758(8168912) WEXITSTATUS 1 Nov 05 10:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7758(8168912) done Nov 05 10:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1921(8163277) WEXITSTATUS 0 Nov 05 10:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1921(8163277) done Nov 05 10:01:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7425(8166214) WEXITSTATUS 1 Nov 05 10:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7425(8166214) done Nov 05 10:01:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2561(8152828) WEXITSTATUS 0 Nov 05 10:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2561(8152828) done Nov 05 10:01:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7600(8167029) WEXITSTATUS 1 Nov 05 10:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7600(8167029) done Nov 05 10:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_8(8165335) WEXITSTATUS 0 Nov 05 10:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_8(8165335) done Nov 05 10:01:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:01:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8169226 NodeList=(null) usec=32071 Nov 05 10:02:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:02:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_457(7571422) WEXITSTATUS 0 Nov 05 10:02:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_457(7571422) done Nov 05 10:02:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169226 NodeList=r907u05n01 #CPUs=10 Partition=pi_fedorov Nov 05 10:02:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:02:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7599(8167028) WEXITSTATUS 1 Nov 05 10:02:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7599(8167028) done Nov 05 10:02:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:02:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_2(8165307) WEXITSTATUS 0 Nov 05 10:02:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_2(8165307) done Nov 05 10:02:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1942(8163402) WEXITSTATUS 0 Nov 05 10:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1942(8163402) done Nov 05 10:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1966(8163694) WEXITSTATUS 0 Nov 05 10:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1966(8163694) done Nov 05 10:02:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169225 WTERMSIG 2 Nov 05 10:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169225 cancelled by interactive user Nov 05 10:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169225 done Nov 05 10:02:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8169225 error Job/step already completing or completed Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_241(8169227) in pi_ohern on r917u13n03 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_242(8169228) in pi_ohern on r917u13n03 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_243(8169229) in pi_ohern on r917u29n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_244(8169230) in pi_ohern on r917u29n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_245(8169231) in pi_ohern on r917u29n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_246(8169232) in pi_ohern on r917u29n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_247(8169233) in pi_ohern on r917u29n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_248(8169234) in pi_ohern on r917u29n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_249(8169235) in pi_ohern on r911u32n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_250(8169236) in pi_ohern on r911u32n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_251(8169237) in pi_ohern on r911u32n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_252(8169238) in pi_ohern on r911u32n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_253(8169239) in pi_ohern on r911u32n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_254(8169240) in pi_ohern on r911u32n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_255(8169241) in pi_ohern on r911u32n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_256(8169242) in pi_ohern on r911u32n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_257(8169243) in pi_ohern on r807u22n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_258(8169244) in pi_ohern on r807u22n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_259(8169245) in pi_ohern on r807u22n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_260(8169246) in pi_ohern on r807u22n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_261(8169247) in pi_ohern on r807u22n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_262(8169248) in pi_ohern on r807u22n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_263(8169249) in pi_ohern on r807u22n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_264(8169250) in pi_ohern on r807u22n01 Nov 05 10:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_265(8169251) in pi_ohern on r807u24n01 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3952(8169252) in gpu on r808u17n01 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3953(8169253) in gpu on r4513u26n01 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2879(8169254) in day on r913u09n04 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2880(8169255) in day on r914u01n04 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2881(8169256) in day on r805u02n04 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2882(8169257) in day on r805u14n02 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2883(8169258) in day on r805u25n04 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2884(8169259) in day on r918u32n03 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2885(8169260) in day on r918u36n01 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2886(8169261) in day on r919u32n02 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2887(8169262) in day on r816u09n07 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_737(7572906) WEXITSTATUS 0 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_737(7572906) done Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_769(8115038) WEXITSTATUS 0 Nov 05 10:02:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_769(8115038) done Nov 05 10:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7750(8168853) WEXITSTATUS 1 Nov 05 10:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7750(8168853) done Nov 05 10:02:32 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8169263 NodeList=(null) usec=1465831 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7787(8169264) in scavenge on r815u29n08 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7788(8169265) in scavenge on r816u23n12 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7789(8169266) in scavenge on r806u06n03 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7790(8169267) in scavenge on r807u11n02 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7791(8169268) in scavenge on r906u41n02 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7792(8169269) in scavenge on r913u09n04 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7793(8169270) in scavenge on r913u09n04 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7794(8169271) in scavenge on r914u01n04 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7795(8169272) in scavenge on r914u01n04 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7796(8169273) in scavenge on r914u01n04 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7797(8169274) in scavenge on r805u02n04 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_79(7571043) in scavenge on r805u06n02 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_504(7572672) in scavenge on r805u25n04 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1023(7573192) in scavenge on r816u09n07 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1107(7570769) WEXITSTATUS 0 Nov 05 10:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1107(7570769) done Nov 05 10:02:36 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169263 NodeList=r807u14n01 #CPUs=8 Partition=pi_panda Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:02:38 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:02:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169275 InitPrio=13509 usec=2630 Nov 05 10:02:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169276 InitPrio=13509 usec=1932 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169277 InitPrio=13509 usec=1972 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169278 InitPrio=13509 usec=1721 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169279 InitPrio=13509 usec=1423 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169280 InitPrio=13509 usec=2740 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169281 InitPrio=13509 usec=1747 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169282 InitPrio=13509 usec=2018 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169283 InitPrio=13509 usec=1548 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169284 InitPrio=13509 usec=2571 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169285 InitPrio=13509 usec=1909 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169286 InitPrio=13509 usec=1484 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169287 InitPrio=13509 usec=1348 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169288 InitPrio=13509 usec=1373 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169289 InitPrio=13509 usec=1304 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169290 InitPrio=13509 usec=1685 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169291 InitPrio=13509 usec=1599 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169292 InitPrio=13509 usec=1582 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169293 InitPrio=13509 usec=1492 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169294 InitPrio=13509 usec=1588 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169295 InitPrio=13509 usec=1363 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169296 InitPrio=13509 usec=1384 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169297 InitPrio=13509 usec=1306 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169298 InitPrio=13509 usec=1316 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169299 InitPrio=13509 usec=1494 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169300 InitPrio=13509 usec=1727 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169301 InitPrio=13509 usec=1523 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169302 InitPrio=13509 usec=1616 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169303 InitPrio=13509 usec=1394 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169304 InitPrio=13509 usec=1445 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169305 InitPrio=13509 usec=1289 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169306 InitPrio=13509 usec=1314 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169307 InitPrio=13509 usec=1244 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169308 InitPrio=13509 usec=1289 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169309 InitPrio=13509 usec=1275 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169310 InitPrio=13509 usec=1635 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169311 InitPrio=13509 usec=1414 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169312 InitPrio=13509 usec=1618 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169313 InitPrio=13509 usec=1462 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169314 InitPrio=13509 usec=1400 Nov 05 10:02:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:02:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3952(8169252) WEXITSTATUS 0 Nov 05 10:02:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3952(8169252) done Nov 05 10:02:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:02:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169315 InitPrio=13750 usec=3561 Nov 05 10:02:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6904(8165151) WEXITSTATUS 1 Nov 05 10:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6904(8165151) done Nov 05 10:03:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169316 InitPrio=13750 usec=5173 Nov 05 10:03:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7748(8168851) WEXITSTATUS 1 Nov 05 10:03:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7748(8168851) done Nov 05 10:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1852(8159661) WEXITSTATUS 0 Nov 05 10:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1852(8159661) done Nov 05 10:03:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3941(8168904) WEXITSTATUS 0 Nov 05 10:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3941(8168904) done Nov 05 10:03:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7412(8166191) WEXITSTATUS 1 Nov 05 10:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7412(8166191) done Nov 05 10:03:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7551(8166748) WEXITSTATUS 1 Nov 05 10:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7551(8166748) done Nov 05 10:03:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7132(8165724) WEXITSTATUS 1 Nov 05 10:03:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7132(8165724) done Nov 05 10:03:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6821(8165000) WEXITSTATUS 1 Nov 05 10:03:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6821(8165000) done Nov 05 10:03:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3953(8169253) WEXITSTATUS 0 Nov 05 10:03:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3953(8169253) done Nov 05 10:03:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6799(8164955) WEXITSTATUS 1 Nov 05 10:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6799(8164955) done Nov 05 10:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7494(8166467) WEXITSTATUS 1 Nov 05 10:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7494(8166467) done Nov 05 10:03:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:03:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7649(8167254) WEXITSTATUS 1 Nov 05 10:03:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7649(8167254) done Nov 05 10:03:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7746(8168849) WEXITSTATUS 1 Nov 05 10:03:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7746(8168849) done Nov 05 10:03:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166472 uid 21442 Nov 05 10:03:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:03:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169315 NodeList=r914u30n01 #CPUs=4 Partition=gpu_devel Nov 05 10:03:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169316 NodeList=r805u30n01 #CPUs=4 Partition=gpu_devel Nov 05 10:03:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7759(8168913) WEXITSTATUS 1 Nov 05 10:04:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7759(8168913) done Nov 05 10:04:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1980(8163894) WEXITSTATUS 0 Nov 05 10:04:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1980(8163894) done Nov 05 10:04:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7489(8166462) WEXITSTATUS 1 Nov 05 10:04:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7489(8166462) done Nov 05 10:04:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7289(8165965) WEXITSTATUS 1 Nov 05 10:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7289(8165965) done Nov 05 10:04:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:04:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_73(8135772) WEXITSTATUS 0 Nov 05 10:04:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_73(8135772) done Nov 05 10:04:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:04:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3946(8169103) WEXITSTATUS 0 Nov 05 10:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3946(8169103) done Nov 05 10:04:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:04:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:04:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3852(8165821) WEXITSTATUS 0 Nov 05 10:04:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3852(8165821) done Nov 05 10:04:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:05:03 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3051(8129627) Nov 05 10:05:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_266(8169317) in pi_ohern on r917u13n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_267(8169318) in pi_ohern on r917u13n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_268(8169319) in pi_ohern on r917u13n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_269(8169320) in pi_ohern on r917u13n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_270(8169321) in pi_ohern on r917u13n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_271(8169322) in pi_ohern on r917u29n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_272(8169323) in pi_ohern on r917u29n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_273(8169324) in pi_ohern on r917u29n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_274(8169325) in pi_ohern on r917u29n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_275(8169326) in pi_ohern on r917u29n03 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_276(8169327) in pi_ohern on r913u32n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_277(8169328) in pi_ohern on r913u32n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_278(8169329) in pi_ohern on r913u32n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_279(8169330) in pi_ohern on r913u32n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_280(8169331) in pi_ohern on r913u32n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_281(8169332) in pi_ohern on r913u32n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_282(8169333) in pi_ohern on r913u32n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_283(8169334) in pi_ohern on r913u32n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3954(8169335) in gpu on r808u08n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3955(8169336) in gpu on r808u08n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3956(8169337) in gpu on r808u14n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3957(8169338) in gpu on r808u17n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3958(8169339) in gpu on r4513u26n01 Nov 05 10:05:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2888(8169340) in day on r918u09n04 Nov 05 10:05:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_752(7572921) WEXITSTATUS 0 Nov 05 10:05:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_752(7572921) done Nov 05 10:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_850(8115119) WEXITSTATUS 0 Nov 05 10:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_850(8115119) done Nov 05 10:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_936(8116410) WEXITSTATUS 0 Nov 05 10:05:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_936(8116410) done Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7798(8169341) in scavenge on r801u06n01 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7799(8169342) in scavenge on r815u03n10 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7800(8169343) in scavenge on r816u15n11 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7801(8169344) in scavenge on r816u29n10 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7802(8169345) in scavenge on r807u11n03 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7803(8169346) in scavenge on r805u06n02 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7804(8169347) in scavenge on r805u27n03 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7805(8169348) in scavenge on r909u01n04 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7806(8169349) in scavenge on r909u10n04 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7807(8169350) in scavenge on r918u01n01 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7808(8169351) in scavenge on r918u09n01 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7809(8169352) in scavenge on r913u09n02 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7810(8169353) in scavenge on r913u24n02 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7811(8169354) in scavenge on r913u28n02 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163239 in scavenge on r801u30n01 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8159213 in scavenge on r801u30n01 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163238 in scavenge on r801u30n01 Nov 05 10:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8163214 in scavenge on r801u30n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169275 in scavenge on r907u09n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169276 in scavenge on r907u09n03 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169277 in scavenge on r802u14n03 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169278 in scavenge on r802u23n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169279 in scavenge on r802u30n04 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169280 in scavenge on r801u06n03 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169281 in scavenge on r801u09n03 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169282 in scavenge on r801u09n04 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169283 in scavenge on r801u27n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169284 in scavenge on r911u09n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169285 in scavenge on r911u09n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169286 in scavenge on r911u09n03 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169287 in scavenge on r911u13n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169288 in scavenge on r911u24n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169289 in scavenge on r911u28n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169290 in scavenge on r911u28n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169291 in scavenge on r912u05n04 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169292 in scavenge on r815u15n09 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169293 in scavenge on r815u15n10 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169294 in scavenge on r815u23n07 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169295 in scavenge on r815u29n09 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169296 in scavenge on r816u23n11 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169297 in scavenge on r816u29n08 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169298 in scavenge on r816u29n11 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169299 in scavenge on r912u13n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169300 in scavenge on r903u34n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169301 in scavenge on r808u17n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169302 in scavenge on r4513u23n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169303 in scavenge on r4513u26n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169304 in scavenge on r807u14n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169305 in scavenge on r902u07n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169306 in scavenge on r906u39n01 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169307 in scavenge on r919u32n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169308 in scavenge on r912u28n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169309 in scavenge on r912u28n03 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169310 in scavenge on r912u28n04 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169311 in scavenge on r912u32n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169312 in scavenge on r912u32n03 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169313 in scavenge on r913u01n02 Nov 05 10:05:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169314 in scavenge on r913u09n03 Nov 05 10:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169305 WEXITSTATUS 1 Nov 05 10:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169305 done Nov 05 10:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169302 WEXITSTATUS 1 Nov 05 10:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169302 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169303 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169303 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169294 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169294 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169297 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169297 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169293 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169293 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169296 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169296 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169298 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169298 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169300 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169300 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169292 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169292 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169304 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169304 done Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169295 WEXITSTATUS 1 Nov 05 10:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169295 done Nov 05 10:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169301 WEXITSTATUS 1 Nov 05 10:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169301 done Nov 05 10:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169275 WEXITSTATUS 1 Nov 05 10:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169275 done Nov 05 10:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169276 WEXITSTATUS 1 Nov 05 10:05:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169276 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7772(8169045) WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7772(8169045) done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169308 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169308 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169307 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169307 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169299 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169299 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169277 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169277 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169280 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169280 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169278 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169278 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169281 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169281 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169314 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169314 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169283 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169283 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169286 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169286 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169309 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169309 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169313 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169313 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169291 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169291 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169306 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169306 done Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169282 WEXITSTATUS 1 Nov 05 10:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169282 done Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169287 WEXITSTATUS 1 Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169287 done Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169310 WEXITSTATUS 1 Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169310 done Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169289 WEXITSTATUS 1 Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169289 done Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169285 WEXITSTATUS 1 Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169285 done Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169284 WEXITSTATUS 1 Nov 05 10:05:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169284 done Nov 05 10:05:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169279 WEXITSTATUS 1 Nov 05 10:05:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169279 done Nov 05 10:05:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169290 WEXITSTATUS 1 Nov 05 10:05:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169290 done Nov 05 10:05:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166801 uid 20354 Nov 05 10:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169288 WEXITSTATUS 1 Nov 05 10:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169288 done Nov 05 10:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169312 WEXITSTATUS 1 Nov 05 10:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169312 done Nov 05 10:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169311 WEXITSTATUS 1 Nov 05 10:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169311 done Nov 05 10:05:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1910(8163230) WEXITSTATUS 0 Nov 05 10:05:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1910(8163230) done Nov 05 10:05:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144147 NodeList=r911u05n01,r917u07n01 #CPUs=84 Partition=scavenge Nov 05 10:05:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 10:05:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 10:05:55 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 10:05:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169355 InitPrio=59702 usec=7964 Nov 05 10:05:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169355 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_1(8158354) Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_2(8158355) Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_3(8158356) Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_4(8158357) Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_5(8158358) Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6860(8165073) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2269(8165100) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2393(8165605) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7436(8166229) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169356 InitPrio=59541 usec=4758 Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1982(8163950) WEXITSTATUS 0 Nov 05 10:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1982(8163950) done Nov 05 10:06:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169356 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 10:06:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6860(8165073) Nov 05 10:06:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2269(8165100) Nov 05 10:06:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2393(8165605) Nov 05 10:06:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7436(8166229) Nov 05 10:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7800(8169343) WEXITSTATUS 1 Nov 05 10:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7800(8169343) done Nov 05 10:06:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7740(8168843) WEXITSTATUS 1 Nov 05 10:06:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7740(8168843) done Nov 05 10:06:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169357 InitPrio=31906 usec=3749 Nov 05 10:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3956(8169337) WEXITSTATUS 0 Nov 05 10:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3956(8169337) done Nov 05 10:06:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:06:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:06:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133379 WEXITSTATUS 0 Nov 05 10:06:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133379 done Nov 05 10:06:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:06:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:06:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1085(7572051) WEXITSTATUS 0 Nov 05 10:06:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1085(7572051) done Nov 05 10:06:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:07:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1973(8163787) WEXITSTATUS 0 Nov 05 10:07:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1973(8163787) done Nov 05 10:07:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3954(8169335) WEXITSTATUS 0 Nov 05 10:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3954(8169335) done Nov 05 10:07:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_284(8169358) in pi_ohern on r917u13n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_285(8169359) in pi_ohern on r917u13n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_286(8169360) in pi_ohern on r917u13n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_287(8169361) in pi_ohern on r917u13n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_288(8169362) in pi_ohern on r917u13n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_289(8169363) in pi_ohern on r917u13n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_290(8169364) in pi_ohern on r917u29n02 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_291(8169365) in pi_ohern on r917u29n02 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_292(8169366) in pi_ohern on r917u29n02 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_293(8169367) in pi_ohern on r917u29n02 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_294(8169368) in pi_ohern on r917u29n02 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_295(8169369) in pi_ohern on r917u29n02 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_296(8169370) in pi_ohern on r917u29n03 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_297(8169371) in pi_ohern on r917u29n03 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_298(8169372) in pi_ohern on r917u29n03 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_299(8169373) in pi_ohern on r917u29n03 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_300(8169374) in pi_ohern on r917u29n03 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_301(8169375) in pi_ohern on r917u29n03 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_302(8169376) in pi_ohern on r815u03n11 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_303(8169377) in pi_ohern on r815u03n11 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_304(8169378) in pi_ohern on r815u03n11 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_305(8169379) in pi_ohern on r815u03n11 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_306(8169380) in pi_ohern on r815u03n12 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_307(8169381) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_308(8169382) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_309(8169383) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_310(8169384) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_311(8169385) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_312(8169386) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_313(8169387) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_314(8169388) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_315(8169389) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_316(8169390) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_317(8169391) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_318(8169392) in pi_ohern on r807u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_319(8169393) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_320(8169394) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_321(8169395) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_322(8169396) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_323(8169397) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_324(8169398) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_325(8169399) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_326(8169400) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_327(8169401) in pi_ohern on r903u24n01 Nov 05 10:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_328(8169402) in pi_ohern on r903u24n01 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169357 in gpu on r816u35n07,r907u32n01 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3959(8169403) in gpu on r808u08n01 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3960(8169404) in gpu on r808u14n01 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u05n03 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2889(8169405) in day on r918u32n02 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2890(8169406) in day on r918u32n03 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2891(8169407) in day on r806u06n04 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2892(8169408) in day on r806u11n04 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2893(8169409) in day on r806u14n02 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2894(8169410) in day on r909u04n01 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2895(8169411) in day on r909u04n03 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2896(8169412) in day on r909u10n03 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166585 in gpu on r915u09n01 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_544(8116017) WEXITSTATUS 0 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8115470_544(8116017) done Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_958(7573127) WEXITSTATUS 0 Nov 05 10:07:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_958(7573127) done Nov 05 10:07:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1462(8138139) WEXITSTATUS 0 Nov 05 10:07:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1462(8138139) done Nov 05 10:07:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169413 InitPrio=22844 usec=5779 Nov 05 10:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7488(8166461) WEXITSTATUS 1 Nov 05 10:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7488(8166461) done Nov 05 10:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7812(8169414) in scavenge on r816u15n11 Nov 05 10:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7813(8169415) in scavenge on r816u23n11 Nov 05 10:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7814(8169416) in scavenge on r909u04n04 Nov 05 10:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7815(8169417) in scavenge on r815u03n07 Nov 05 10:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7816(8169418) in scavenge on r815u03n07 Nov 05 10:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7817(8169419) in scavenge on r815u03n08 Nov 05 10:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169357 WEXITSTATUS 1 Nov 05 10:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169357 done Nov 05 10:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154691 WEXITSTATUS 0 Nov 05 10:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154691 done Nov 05 10:08:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169420 InitPrio=13300 usec=7307 Nov 05 10:08:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3937(8168714) WEXITSTATUS 0 Nov 05 10:08:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3937(8168714) done Nov 05 10:08:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:08:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6873(8165096) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:08:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2289(8165152) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:08:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6873(8165096) Nov 05 10:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1946(8163442) WEXITSTATUS 0 Nov 05 10:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1946(8163442) done Nov 05 10:08:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2289(8165152) Nov 05 10:08:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651307_3(7651773) Nov 05 10:08:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:08:37 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:08:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7537(8166669) WEXITSTATUS 1 Nov 05 10:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7537(8166669) done Nov 05 10:08:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:08:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_17(8168810) WEXITSTATUS 0 Nov 05 10:08:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_17(8168810) done Nov 05 10:08:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:08:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:08:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169420 NodeList=r903u37n01 #CPUs=6 Partition=gpu_devel Nov 05 10:08:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7660(8167612) WEXITSTATUS 1 Nov 05 10:08:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7660(8167612) done Nov 05 10:08:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7804(8169347) WEXITSTATUS 1 Nov 05 10:08:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7804(8169347) done Nov 05 10:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_18(8168811) WEXITSTATUS 0 Nov 05 10:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_18(8168811) done Nov 05 10:08:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:08:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7503(8166495) WEXITSTATUS 1 Nov 05 10:08:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7503(8166495) done Nov 05 10:08:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:08:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169421 InitPrio=51880 usec=7229 Nov 05 10:08:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169421 NodeList=r917u23n01 #CPUs=8 Partition=pi_lederman Nov 05 10:08:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_6(8158375) Nov 05 10:09:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_7(8158376) Nov 05 10:09:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_8(8158377) Nov 05 10:09:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_9(8158378) Nov 05 10:09:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_10(8158379) Nov 05 10:09:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_11(8158380) Nov 05 10:09:05 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:09:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3958(8169339) WEXITSTATUS 0 Nov 05 10:09:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3958(8169339) done Nov 05 10:09:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7162(8165773) WEXITSTATUS 1 Nov 05 10:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7162(8165773) done Nov 05 10:09:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168537 uid 22479 Nov 05 10:09:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_19(8168812) WEXITSTATUS 0 Nov 05 10:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_19(8168812) done Nov 05 10:09:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7067(8165560) WEXITSTATUS 1 Nov 05 10:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7067(8165560) done Nov 05 10:09:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167619 WEXITSTATUS 0 Nov 05 10:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167619 done Nov 05 10:09:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_20(8168813) WEXITSTATUS 0 Nov 05 10:09:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_20(8168813) done Nov 05 10:09:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:09:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144148 NodeList=r801u23n02,r907u36n01 #CPUs=72 Partition=scavenge Nov 05 10:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1941(8163401) WEXITSTATUS 0 Nov 05 10:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1941(8163401) done Nov 05 10:09:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7790(8169267) WEXITSTATUS 1 Nov 05 10:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7790(8169267) done Nov 05 10:10:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3960(8169404) WEXITSTATUS 0 Nov 05 10:10:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3960(8169404) done Nov 05 10:10:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160533 WEXITSTATUS 0 Nov 05 10:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160533 done Nov 05 10:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3947(8169104) WEXITSTATUS 0 Nov 05 10:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3947(8169104) done Nov 05 10:10:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_329(8169422) in pi_ohern on r801u32n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_330(8169423) in pi_ohern on r801u32n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_331(8169424) in pi_ohern on r801u32n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_332(8169425) in pi_ohern on r801u32n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_333(8169426) in pi_ohern on r801u32n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_334(8169427) in pi_ohern on r801u32n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_335(8169428) in pi_ohern on r917u13n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_336(8169429) in pi_ohern on r917u13n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_337(8169430) in pi_ohern on r917u13n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_338(8169431) in pi_ohern on r917u13n04 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_339(8169432) in pi_ohern on r917u13n04 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_340(8169433) in pi_ohern on r917u13n04 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_341(8169434) in pi_ohern on r917u13n04 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_342(8169435) in pi_ohern on r917u13n04 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_343(8169436) in pi_ohern on r815u03n11 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_344(8169437) in pi_ohern on r815u03n11 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_345(8169438) in pi_ohern on r815u03n11 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_346(8169439) in pi_ohern on r815u03n11 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_347(8169440) in pi_ohern on r815u03n11 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_348(8169441) in pi_ohern on r815u03n12 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169413 in gpu on r808u11n01 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3961(8169442) in gpu on r808u14n01 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3962(8169443) in gpu on r808u14n01 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3963(8169444) in gpu on r4513u26n01 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3964(8169445) in gpu on r907u32n01 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2897(8169446) in day on r805u06n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2898(8169447) in day on r913u13n04 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2899(8169448) in day on r806u09n01 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2900(8169449) in day on r806u11n03 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2901(8169450) in day on r918u09n01 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2902(8169451) in day on r918u36n03 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2903(8169452) in day on r919u09n03 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2904(8169453) in day on r816u15n02 Nov 05 10:10:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2905(8169454) in day on r816u15n02 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7818(8169455) in scavenge on r801u14n01 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7819(8169456) in scavenge on r911u01n01 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7820(8169457) in scavenge on r816u15n09 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7821(8169458) in scavenge on r805u27n03 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7822(8169459) in scavenge on r913u13n04 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7823(8169460) in scavenge on r807u08n02 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7824(8169461) in scavenge on r807u11n02 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7825(8169462) in scavenge on r918u09n01 Nov 05 10:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_144(8114410) in scavenge on r918u36n03 Nov 05 10:10:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2269(8165100) in scavenge on r801u06n01 Nov 05 10:10:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2393(8165605) in scavenge on r816u29n08 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:10:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133378 WEXITSTATUS 0 Nov 05 10:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133378 done Nov 05 10:11:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168538 WEXITSTATUS 0 Nov 05 10:11:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168538 done Nov 05 10:11:06 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2268(8165099) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:11:06 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6927(8165199) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:11:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6927(8165199) Nov 05 10:11:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2268(8165099) Nov 05 10:11:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169463 InitPrio=90911 usec=7550 Nov 05 10:11:09 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169463 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 10:11:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_21(8168814) WEXITSTATUS 0 Nov 05 10:11:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_21(8168814) done Nov 05 10:11:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8168557 uid 26133 Nov 05 10:11:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1989(8164088) WEXITSTATUS 0 Nov 05 10:11:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1989(8164088) done Nov 05 10:11:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7129(8165712) WEXITSTATUS 1 Nov 05 10:11:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7129(8165712) done Nov 05 10:11:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7602(8167031) WEXITSTATUS 1 Nov 05 10:11:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7602(8167031) done Nov 05 10:11:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168559 WEXITSTATUS 0 Nov 05 10:11:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168559 done Nov 05 10:11:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_12(8158396) Nov 05 10:11:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_13(8158397) Nov 05 10:11:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_14(8158398) Nov 05 10:11:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_15(8158399) Nov 05 10:11:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_16(8158400) Nov 05 10:11:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_17(8158401) Nov 05 10:11:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_18(8158402) Nov 05 10:11:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_19(8158403) Nov 05 10:11:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169464 InitPrio=34739 usec=2281 Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7651307_3(7651773) NodeList=r815u35n07 #CPUs=1 Partition=scavenge_gpu Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_0(8169465) NodeList=r912u01n01 #CPUs=8 Partition=scavenge Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_1(8169466) NodeList=r801u14n03 #CPUs=8 Partition=scavenge Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_2(8169467) NodeList=r801u25n01 #CPUs=8 Partition=scavenge Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_3(8169468) NodeList=r903u01n02 #CPUs=8 Partition=scavenge Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_4(8169469) NodeList=r903u01n03 #CPUs=8 Partition=scavenge Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_5(8169470) NodeList=r903u04n01 #CPUs=8 Partition=scavenge Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_6(8169471) NodeList=r903u04n01 #CPUs=8 Partition=scavenge Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_7(8169472) NodeList=r904u01n01 #CPUs=8 Partition=scavenge Nov 05 10:11:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169464_8(8169464) NodeList=r904u01n01 #CPUs=8 Partition=scavenge Nov 05 10:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154716 WEXITSTATUS 0 Nov 05 10:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154716 done Nov 05 10:11:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7187(8165812) WEXITSTATUS 1 Nov 05 10:11:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7187(8165812) done Nov 05 10:12:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8169464 uid 22757 Nov 05 10:12:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7095(8165651) WEXITSTATUS 1 Nov 05 10:12:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7095(8165651) done Nov 05 10:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7777(8169116) WEXITSTATUS 1 Nov 05 10:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7777(8169116) done Nov 05 10:12:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6535(8164480) WEXITSTATUS 1 Nov 05 10:12:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6535(8164480) done Nov 05 10:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166567 WEXITSTATUS 0 Nov 05 10:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166567 done Nov 05 10:12:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169473 InitPrio=10016 usec=4098 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:12:39 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169473 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 10:12:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169474 InitPrio=34739 usec=2898 Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_0(8169475) NodeList=r912u01n01 #CPUs=8 Partition=scavenge Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_1(8169476) NodeList=r801u14n03 #CPUs=8 Partition=scavenge Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_2(8169477) NodeList=r801u25n01 #CPUs=8 Partition=scavenge Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_3(8169478) NodeList=r903u01n02 #CPUs=8 Partition=scavenge Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_4(8169479) NodeList=r903u01n03 #CPUs=8 Partition=scavenge Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_5(8169480) NodeList=r903u04n01 #CPUs=8 Partition=scavenge Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_6(8169481) NodeList=r903u04n01 #CPUs=8 Partition=scavenge Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_7(8169482) NodeList=r904u01n01 #CPUs=8 Partition=scavenge Nov 05 10:12:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169474_8(8169474) NodeList=r904u01n01 #CPUs=8 Partition=scavenge Nov 05 10:13:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8169463 uid 26587 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_349(8169483) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_350(8169484) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_351(8169485) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_352(8169486) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_353(8169487) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_354(8169488) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_355(8169489) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_356(8169490) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_357(8169491) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_358(8169492) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_359(8169493) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_360(8169494) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_361(8169495) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_362(8169496) in pi_ohern on r815u03n11 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_363(8169497) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_364(8169498) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_365(8169499) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_366(8169500) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_367(8169501) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_368(8169502) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_369(8169503) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_370(8169504) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_371(8169505) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_372(8169506) in pi_ohern on r815u03n12 Nov 05 10:13:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_373(8169507) in pi_ohern on r815u03n12 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168540 in day on r805u23n02 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n03 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2906(8169508) in day on r806u11n01 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2907(8169509) in day on r914u05n01 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2908(8169510) in day on r807u05n02 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2909(8169511) in day on r807u05n03 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2910(8169512) in day on r909u01n02 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2911(8169513) in day on r909u07n04 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2912(8169514) in day on r909u10n02 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2913(8169515) in day on r909u13n03 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2914(8169516) in day on r909u29n02 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2915(8169517) in day on r815u15n03 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2916(8169518) in day on r815u23n04 Nov 05 10:13:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166586 in gpu on r4513u32n01 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7826(8169519) in scavenge on r802u06n04 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7827(8169520) in scavenge on r801u14n03 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7828(8169521) in scavenge on r801u25n01 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7829(8169522) in scavenge on r801u25n01 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7830(8169523) in scavenge on r815u35n07 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7831(8169524) in scavenge on r918u09n01 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7832(8169525) in scavenge on r807u05n01 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_611(8114878) in scavenge on r807u05n03 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_863(8115132) in scavenge on r815u23n04 Nov 05 10:13:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_984(7573153) in scavenge on r904u13n01 Nov 05 10:13:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2289(8165152) in scavenge on r903u04n01 Nov 05 10:13:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7723(8168549) WEXITSTATUS 1 Nov 05 10:13:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7723(8168549) done Nov 05 10:13:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1925(8163304) WEXITSTATUS 0 Nov 05 10:13:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1925(8163304) done Nov 05 10:13:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651307_5(7651948) Nov 05 10:13:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7814(8169416) WEXITSTATUS 1 Nov 05 10:13:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7814(8169416) done Nov 05 10:13:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1969(8163740) WEXITSTATUS 0 Nov 05 10:13:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1969(8163740) done Nov 05 10:13:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7578(8166816) WEXITSTATUS 1 Nov 05 10:13:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7578(8166816) done Nov 05 10:13:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_24(8168817) WEXITSTATUS 0 Nov 05 10:13:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_24(8168817) done Nov 05 10:13:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7585(8167000) WEXITSTATUS 1 Nov 05 10:13:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7585(8167000) done Nov 05 10:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_43(8168836) WEXITSTATUS 0 Nov 05 10:13:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_43(8168836) done Nov 05 10:14:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2298(8165184) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:14:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2370(8165577) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:14:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2298(8165184) Nov 05 10:14:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2370(8165577) Nov 05 10:14:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169526 InitPrio=26063 usec=2039 Nov 05 10:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7579(8166817) WEXITSTATUS 1 Nov 05 10:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7579(8166817) done Nov 05 10:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6813(8164985) WEXITSTATUS 1 Nov 05 10:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6813(8164985) done Nov 05 10:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7831(8169524) WEXITSTATUS 1 Nov 05 10:14:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7831(8169524) done Nov 05 10:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_22(8168815) WEXITSTATUS 0 Nov 05 10:14:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_22(8168815) done Nov 05 10:14:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_4(8168797) WEXITSTATUS 0 Nov 05 10:14:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_4(8168797) done Nov 05 10:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_13(8168806) WEXITSTATUS 0 Nov 05 10:14:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_13(8168806) done Nov 05 10:14:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7549(8166746) WEXITSTATUS 1 Nov 05 10:14:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7549(8166746) done Nov 05 10:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7525(8166544) WEXITSTATUS 1 Nov 05 10:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7525(8166544) done Nov 05 10:14:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8158160_20(8158160) Nov 05 10:14:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:14:39 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:14:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7798(8169341) WEXITSTATUS 1 Nov 05 10:14:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7798(8169341) done Nov 05 10:14:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_9(8168802) WEXITSTATUS 0 Nov 05 10:14:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_9(8168802) done Nov 05 10:14:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_41(8168834) WEXITSTATUS 0 Nov 05 10:14:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_41(8168834) done Nov 05 10:14:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169526 NodeList=r805u30n01 #CPUs=2 Partition=gpu_devel Nov 05 10:14:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1851(8159396) WEXITSTATUS 0 Nov 05 10:14:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1851(8159396) done Nov 05 10:14:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6718(8164829) WEXITSTATUS 1 Nov 05 10:14:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6718(8164829) done Nov 05 10:14:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_23(8168816) WEXITSTATUS 0 Nov 05 10:14:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_23(8168816) done Nov 05 10:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_42(8168835) WEXITSTATUS 0 Nov 05 10:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_42(8168835) done Nov 05 10:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_10(8168803) WEXITSTATUS 0 Nov 05 10:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_10(8168803) done Nov 05 10:15:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_33(8168826) WEXITSTATUS 0 Nov 05 10:15:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_33(8168826) done Nov 05 10:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_15(8168808) WEXITSTATUS 0 Nov 05 10:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_15(8168808) done Nov 05 10:15:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1997(8164173) WEXITSTATUS 0 Nov 05 10:15:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1997(8164173) done Nov 05 10:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7465(8166319) WEXITSTATUS 1 Nov 05 10:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7465(8166319) done Nov 05 10:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3912(8166789) WEXITSTATUS 0 Nov 05 10:15:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3912(8166789) done Nov 05 10:15:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7478(8166435) WEXITSTATUS 1 Nov 05 10:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7478(8166435) done Nov 05 10:15:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_12(8168805) WEXITSTATUS 0 Nov 05 10:15:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_12(8168805) done Nov 05 10:15:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_8(8168801) WEXITSTATUS 0 Nov 05 10:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_8(8168801) done Nov 05 10:15:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_374(8169527) in pi_ohern on r917u13n03 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_375(8169528) in pi_ohern on r917u13n04 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_376(8169529) in pi_ohern on r917u13n04 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_377(8169530) in pi_ohern on r917u13n04 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_378(8169531) in pi_ohern on r917u29n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_379(8169532) in pi_ohern on r917u29n02 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_380(8169533) in pi_ohern on r917u29n04 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_381(8169534) in pi_ohern on r815u03n11 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_382(8169535) in pi_ohern on r815u03n11 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_383(8169536) in pi_ohern on r815u03n11 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_384(8169537) in pi_ohern on r815u03n12 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_385(8169538) in pi_ohern on r815u03n12 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_386(8169539) in pi_ohern on r903u12n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_387(8169540) in pi_ohern on r903u12n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_388(8169541) in pi_ohern on r903u12n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_389(8169542) in pi_ohern on r911u32n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_390(8169543) in pi_ohern on r911u32n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_391(8169544) in pi_ohern on r913u32n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3965(8169545) in gpu on r808u14n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7651307_5(7651948) in gpu on r907u34n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2917(8169546) in day on r805u11n04 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2918(8169547) in day on r909u29n03 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2919(8169548) in day on r914u01n01 Nov 05 10:15:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2920(8169549) in day on r816u09n05 Nov 05 10:15:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_30(8168823) WEXITSTATUS 0 Nov 05 10:15:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_30(8168823) done Nov 05 10:15:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_55(8168887) WEXITSTATUS 0 Nov 05 10:15:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_55(8168887) done Nov 05 10:15:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7140(8165732) WEXITSTATUS 1 Nov 05 10:15:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7140(8165732) done Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7833(8169550) in scavenge on r801u06n01 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7834(8169551) in scavenge on r802u23n02 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7835(8169552) in scavenge on r4513u26n01 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7836(8169553) in scavenge on r805u06n04 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7837(8169554) in scavenge on r805u11n04 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7838(8169555) in scavenge on r807u05n03 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7839(8169556) in scavenge on r807u05n03 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7840(8169557) in scavenge on r807u11n01 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7841(8169558) in scavenge on r913u24n04 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7842(8169559) in scavenge on r913u28n01 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7843(8169560) in scavenge on r918u05n03 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7844(8169561) in scavenge on r918u05n03 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7845(8169562) in scavenge on r918u05n04 Nov 05 10:15:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7846(8169563) in scavenge on r918u09n01 Nov 05 10:15:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2268(8165099) in scavenge on r816u29n08 Nov 05 10:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7764(8169037) WEXITSTATUS 1 Nov 05 10:16:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7764(8169037) done Nov 05 10:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3961(8169442) WEXITSTATUS 0 Nov 05 10:16:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3961(8169442) done Nov 05 10:16:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_1(8168794) WEXITSTATUS 0 Nov 05 10:16:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_1(8168794) done Nov 05 10:16:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7835(8169552) WEXITSTATUS 1 Nov 05 10:16:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7835(8169552) done Nov 05 10:16:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_31(8168824) WEXITSTATUS 0 Nov 05 10:16:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_31(8168824) done Nov 05 10:16:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_53(8168885) WEXITSTATUS 0 Nov 05 10:16:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_53(8168885) done Nov 05 10:16:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169564 InitPrio=16342 usec=3944 Nov 05 10:16:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169565 InitPrio=16342 usec=5347 Nov 05 10:16:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169566 InitPrio=16342 usec=4625 Nov 05 10:16:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169567 InitPrio=16342 usec=3467 Nov 05 10:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_44(8168837) WEXITSTATUS 0 Nov 05 10:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_44(8168837) done Nov 05 10:16:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_14(8168807) WEXITSTATUS 0 Nov 05 10:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_14(8168807) done Nov 05 10:16:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_11(8168804) WEXITSTATUS 0 Nov 05 10:16:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_11(8168804) done Nov 05 10:16:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7171(8165789) WEXITSTATUS 1 Nov 05 10:16:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7171(8165789) done Nov 05 10:16:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:36 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7072(8165565) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:16:36 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7305(8165989) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:16:36 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2635(8166298) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:16:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7072(8165565) Nov 05 10:16:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2635(8166298) Nov 05 10:16:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:39 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7305(8165989) Nov 05 10:16:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_34(8168827) WEXITSTATUS 0 Nov 05 10:16:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_34(8168827) done Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:16:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:16:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_2(8168795) WEXITSTATUS 0 Nov 05 10:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_2(8168795) done Nov 05 10:16:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7815(8169417) WEXITSTATUS 1 Nov 05 10:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7815(8169417) done Nov 05 10:16:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3962(8169443) WEXITSTATUS 0 Nov 05 10:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3962(8169443) done Nov 05 10:16:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169568 InitPrio=58914 usec=7849 Nov 05 10:16:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_29(8168822) WEXITSTATUS 0 Nov 05 10:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_29(8168822) done Nov 05 10:17:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_58(8168890) WEXITSTATUS 0 Nov 05 10:17:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_58(8168890) done Nov 05 10:17:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154672 WEXITSTATUS 0 Nov 05 10:17:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154672 done Nov 05 10:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_38(8168831) WEXITSTATUS 0 Nov 05 10:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_38(8168831) done Nov 05 10:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_40(8168833) WEXITSTATUS 0 Nov 05 10:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_40(8168833) done Nov 05 10:17:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7351(8166060) WEXITSTATUS 1 Nov 05 10:17:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7351(8166060) done Nov 05 10:17:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_56(8168888) WEXITSTATUS 0 Nov 05 10:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_56(8168888) done Nov 05 10:17:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_3(8168796) WEXITSTATUS 0 Nov 05 10:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_3(8168796) done Nov 05 10:17:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_32(8168825) WEXITSTATUS 0 Nov 05 10:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_32(8168825) done Nov 05 10:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7032(8165457) WEXITSTATUS 1 Nov 05 10:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7032(8165457) done Nov 05 10:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1922(8163278) WEXITSTATUS 0 Nov 05 10:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1922(8163278) done Nov 05 10:17:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_36(8168829) WEXITSTATUS 0 Nov 05 10:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_36(8168829) done Nov 05 10:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_59(8168891) WEXITSTATUS 0 Nov 05 10:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_59(8168891) done Nov 05 10:17:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_6(8168799) WEXITSTATUS 0 Nov 05 10:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_6(8168799) done Nov 05 10:17:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154658 WEXITSTATUS 0 Nov 05 10:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154658 done Nov 05 10:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168539 WEXITSTATUS 0 Nov 05 10:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168539 done Nov 05 10:17:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7714(8168527) WEXITSTATUS 1 Nov 05 10:17:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7714(8168527) done Nov 05 10:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_7(8168800) WEXITSTATUS 0 Nov 05 10:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_7(8168800) done Nov 05 10:17:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_5(8168798) WEXITSTATUS 0 Nov 05 10:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_5(8168798) done Nov 05 10:17:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169568 NodeList=r913u01n01 #CPUs=8 Partition=day Nov 05 10:17:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_26(8168819) WEXITSTATUS 0 Nov 05 10:17:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_26(8168819) done Nov 05 10:17:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_51(8168883) WEXITSTATUS 0 Nov 05 10:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_51(8168883) done Nov 05 10:18:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_45(8168838) WEXITSTATUS 0 Nov 05 10:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_45(8168838) done Nov 05 10:18:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169569 InitPrio=28268 usec=3362 Nov 05 10:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_54(8168886) WEXITSTATUS 0 Nov 05 10:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_54(8168886) done Nov 05 10:18:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_62(8168894) WEXITSTATUS 0 Nov 05 10:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_62(8168894) done Nov 05 10:18:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:18:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8169569 uid 26133 Nov 05 10:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_61(8168893) WEXITSTATUS 0 Nov 05 10:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_61(8168893) done Nov 05 10:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_63(8168895) WEXITSTATUS 0 Nov 05 10:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_63(8168895) done Nov 05 10:18:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7445(8166274) WEXITSTATUS 1 Nov 05 10:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7445(8166274) done Nov 05 10:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7128(8165711) WEXITSTATUS 1 Nov 05 10:18:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7128(8165711) done Nov 05 10:18:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:18:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169570 InitPrio=28268 usec=3444 Nov 05 10:18:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:18:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_57(8168889) WEXITSTATUS 0 Nov 05 10:18:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_57(8168889) done Nov 05 10:18:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:18:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6772(8164921) WEXITSTATUS 1 Nov 05 10:18:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6772(8164921) done Nov 05 10:18:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7788(8169265) WEXITSTATUS 1 Nov 05 10:18:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7788(8169265) done Nov 05 10:18:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_392(8169571) in pi_ohern on r801u32n02 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_393(8169572) in pi_ohern on r917u13n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_394(8169573) in pi_ohern on r917u13n03 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_395(8169574) in pi_ohern on r917u13n04 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_396(8169575) in pi_ohern on r917u13n04 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_397(8169576) in pi_ohern on r917u13n04 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_398(8169577) in pi_ohern on r917u29n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_399(8169578) in pi_ohern on r917u29n02 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_400(8169579) in pi_ohern on r917u29n04 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_401(8169580) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_402(8169581) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_403(8169582) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_404(8169583) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_405(8169584) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_406(8169585) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_407(8169586) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_408(8169587) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_409(8169588) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_410(8169589) in pi_ohern on r815u03n11 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_411(8169590) in pi_ohern on r815u03n12 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_412(8169591) in pi_ohern on r911u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_413(8169592) in pi_ohern on r911u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_414(8169593) in pi_ohern on r911u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_415(8169594) in pi_ohern on r911u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_416(8169595) in pi_ohern on r911u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_417(8169596) in pi_ohern on r911u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_418(8169597) in pi_ohern on r911u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_419(8169598) in pi_ohern on r911u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_420(8169599) in pi_ohern on r807u22n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_421(8169600) in pi_ohern on r807u22n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_422(8169601) in pi_ohern on r807u24n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_423(8169602) in pi_ohern on r913u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_424(8169603) in pi_ohern on r913u32n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_425(8169604) in pi_ohern on r913u35n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_426(8169605) in pi_ohern on r913u35n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_427(8169606) in pi_ohern on r903u12n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_428(8169607) in pi_ohern on r903u24n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_429(8169608) in pi_ohern on r903u24n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_430(8169609) in pi_ohern on r903u24n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_431(8169610) in pi_ohern on r903u24n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_432(8169611) in pi_ohern on r903u24n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_433(8169612) in pi_ohern on r903u26n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3966(8169613) in gpu on r808u14n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3967(8169614) in gpu on r808u14n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168541 in day on r919u32n04 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169564 in day on r919u09n01 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169567 in day on r815u35n03 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n03 Nov 05 10:18:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2921(8169615) in day on r815u35n06 Nov 05 10:18:31 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:112 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 10:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_67(8168899) WEXITSTATUS 0 Nov 05 10:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_67(8168899) done Nov 05 10:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_104(8168961) WEXITSTATUS 0 Nov 05 10:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_104(8168961) done Nov 05 10:18:35 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8167103 Nov 05 10:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154684 WEXITSTATUS 0 Nov 05 10:18:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154684 done Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7847(8169616) in scavenge on r801u06n03 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7848(8169617) in scavenge on r904u01n03 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7849(8169618) in scavenge on r815u23n08 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7850(8169619) in scavenge on r816u23n12 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7851(8169620) in scavenge on r4513u26n01 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7852(8169621) in scavenge on r903u40n01 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7853(8169622) in scavenge on r805u25n03 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7854(8169623) in scavenge on r807u11n01 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7855(8169624) in scavenge on r912u28n03 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7856(8169625) in scavenge on r918u01n03 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7857(8169626) in scavenge on r909u01n02 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7858(8169627) in scavenge on r909u01n04 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7859(8169628) in scavenge on r909u04n04 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_231(8114497) in scavenge on r815u35n03 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_649(8114918) in scavenge on r815u35n03 Nov 05 10:18:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1114(7573283) in scavenge on r815u35n06 Nov 05 10:18:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_69(8168901) WEXITSTATUS 0 Nov 05 10:18:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_69(8168901) done Nov 05 10:18:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2298(8165184) in scavenge on r912u13n01 Nov 05 10:18:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2370(8165577) in scavenge on r912u13n01 Nov 05 10:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7577(8166815) WEXITSTATUS 1 Nov 05 10:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7577(8166815) done Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:18:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_86(8168943) WEXITSTATUS 0 Nov 05 10:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_86(8168943) done Nov 05 10:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_16(8168809) WEXITSTATUS 0 Nov 05 10:18:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_16(8168809) done Nov 05 10:18:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169629 InitPrio=17452 usec=3263 Nov 05 10:18:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169629 NodeList=r806u23n04 #CPUs=2 Partition=devel Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169630 InitPrio=13509 usec=2380 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169631 InitPrio=13509 usec=1820 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169632 InitPrio=13509 usec=2757 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169633 InitPrio=13509 usec=2268 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169634 InitPrio=13509 usec=1554 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169635 InitPrio=13509 usec=1354 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169636 InitPrio=13509 usec=1359 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169637 InitPrio=13509 usec=1348 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169638 InitPrio=13509 usec=1289 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169639 InitPrio=13509 usec=1328 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169640 InitPrio=13509 usec=1301 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169641 InitPrio=13509 usec=1390 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169642 InitPrio=13509 usec=1340 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169643 InitPrio=13509 usec=1307 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169644 InitPrio=13509 usec=2271 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169645 InitPrio=13509 usec=1609 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169646 InitPrio=13509 usec=1596 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169647 InitPrio=13509 usec=1293 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169648 InitPrio=13509 usec=2210 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169649 InitPrio=13509 usec=1669 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169650 InitPrio=13509 usec=2687 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169651 InitPrio=13509 usec=2068 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169652 InitPrio=13509 usec=1553 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169653 InitPrio=13509 usec=2392 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169654 InitPrio=13509 usec=1856 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169655 InitPrio=13509 usec=1533 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169656 InitPrio=13509 usec=1455 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169657 InitPrio=13509 usec=1465 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169658 InitPrio=13509 usec=1929 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169659 InitPrio=13509 usec=1521 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169660 InitPrio=13509 usec=1365 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169661 InitPrio=13509 usec=1453 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169662 InitPrio=13509 usec=1369 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169663 InitPrio=13509 usec=2111 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169664 InitPrio=13509 usec=1499 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169665 InitPrio=13509 usec=2322 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169666 InitPrio=13509 usec=1710 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169667 InitPrio=13509 usec=1411 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169668 InitPrio=13509 usec=1730 Nov 05 10:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169669 InitPrio=13509 usec=1469 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169670 InitPrio=13509 usec=1494 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169671 InitPrio=13509 usec=2130 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169672 InitPrio=13509 usec=1732 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169673 InitPrio=13509 usec=2031 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169674 InitPrio=13509 usec=1473 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169675 InitPrio=13509 usec=1504 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169676 InitPrio=13509 usec=1465 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169677 InitPrio=13509 usec=1388 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169678 InitPrio=13509 usec=1761 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169679 InitPrio=13509 usec=1487 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169680 InitPrio=13509 usec=1365 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169681 InitPrio=13509 usec=1528 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169682 InitPrio=13509 usec=1859 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169683 InitPrio=13509 usec=1617 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169684 InitPrio=13509 usec=1578 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169685 InitPrio=13509 usec=2729 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169686 InitPrio=13509 usec=2029 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7853(8169622) WEXITSTATUS 1 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7853(8169622) done Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169687 InitPrio=13509 usec=1797 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169688 InitPrio=13509 usec=2077 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169689 InitPrio=13509 usec=1667 Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3966(8169613) WEXITSTATUS 0 Nov 05 10:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3966(8169613) done Nov 05 10:18:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_28(8168821) WEXITSTATUS 0 Nov 05 10:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_28(8168821) done Nov 05 10:19:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_85(8168942) WEXITSTATUS 0 Nov 05 10:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_85(8168942) done Nov 05 10:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_39(8168832) WEXITSTATUS 0 Nov 05 10:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_39(8168832) done Nov 05 10:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_52(8168884) WEXITSTATUS 0 Nov 05 10:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_52(8168884) done Nov 05 10:19:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=7651307_0(7651367) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:19:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2368(8165575) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:19:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2369(8165576) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:19:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2391(8165603) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:19:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7358(8166074) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:19:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2597(8166234) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:19:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7589(8167004) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:19:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7358(8166074) Nov 05 10:19:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7589(8167004) Nov 05 10:19:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2597(8166234) Nov 05 10:19:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651307_0(7651367) Nov 05 10:19:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2391(8165603) Nov 05 10:19:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2368(8165575) Nov 05 10:19:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2369(8165576) Nov 05 10:19:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_66(8168898) WEXITSTATUS 0 Nov 05 10:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_66(8168898) done Nov 05 10:19:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_92(8168949) WEXITSTATUS 0 Nov 05 10:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_92(8168949) done Nov 05 10:19:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_127(8168984) WEXITSTATUS 0 Nov 05 10:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_127(8168984) done Nov 05 10:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_35(8168828) WEXITSTATUS 0 Nov 05 10:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_35(8168828) done Nov 05 10:19:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1995(8164171) WEXITSTATUS 0 Nov 05 10:19:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1995(8164171) done Nov 05 10:19:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_89(8168946) WEXITSTATUS 0 Nov 05 10:19:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_89(8168946) done Nov 05 10:19:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133380 WEXITSTATUS 0 Nov 05 10:19:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133380 done Nov 05 10:19:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_27(8168820) WEXITSTATUS 0 Nov 05 10:19:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_27(8168820) done Nov 05 10:19:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:19:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_64(8168896) WEXITSTATUS 0 Nov 05 10:19:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_64(8168896) done Nov 05 10:19:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_334(8169427) WEXITSTATUS 0 Nov 05 10:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_334(8169427) done Nov 05 10:19:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_120(8168977) WEXITSTATUS 0 Nov 05 10:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_120(8168977) done Nov 05 10:19:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169570 NodeList=r815u35n07 #CPUs=4 Partition=gpu_devel Nov 05 10:19:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_125(8168982) WEXITSTATUS 0 Nov 05 10:19:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_125(8168982) done Nov 05 10:19:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_95(8168952) WEXITSTATUS 0 Nov 05 10:19:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_95(8168952) done Nov 05 10:19:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:19:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7801(8169344) WEXITSTATUS 1 Nov 05 10:19:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7801(8169344) done Nov 05 10:20:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5736(8162592) WEXITSTATUS 1 Nov 05 10:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5736(8162592) done Nov 05 10:20:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_82(8168939) WEXITSTATUS 0 Nov 05 10:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_82(8168939) done Nov 05 10:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7712(8168525) WEXITSTATUS 1 Nov 05 10:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7712(8168525) done Nov 05 10:20:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7218(8165864) WEXITSTATUS 1 Nov 05 10:20:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7218(8165864) done Nov 05 10:20:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_68(8168900) WEXITSTATUS 0 Nov 05 10:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_68(8168900) done Nov 05 10:20:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169690 InitPrio=31906 usec=4987 Nov 05 10:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_91(8168948) WEXITSTATUS 0 Nov 05 10:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_91(8168948) done Nov 05 10:20:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169691 InitPrio=20039 usec=3774 Nov 05 10:20:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168877 WEXITSTATUS 0 Nov 05 10:20:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168877 done Nov 05 10:20:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_46(8168878) WEXITSTATUS 0 Nov 05 10:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_46(8168878) done Nov 05 10:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_123(8168980) WEXITSTATUS 0 Nov 05 10:20:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_123(8168980) done Nov 05 10:20:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_84(8168941) WEXITSTATUS 0 Nov 05 10:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_84(8168941) done Nov 05 10:20:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_114(8168971) WEXITSTATUS 0 Nov 05 10:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_114(8168971) done Nov 05 10:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_145(8169062) WEXITSTATUS 0 Nov 05 10:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_145(8169062) done Nov 05 10:20:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_90(8168947) WEXITSTATUS 0 Nov 05 10:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_90(8168947) done Nov 05 10:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7825(8169462) WEXITSTATUS 1 Nov 05 10:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7825(8169462) done Nov 05 10:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_80(8168937) WEXITSTATUS 0 Nov 05 10:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_80(8168937) done Nov 05 10:20:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_83(8168940) WEXITSTATUS 0 Nov 05 10:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_83(8168940) done Nov 05 10:20:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_79(8168936) WEXITSTATUS 0 Nov 05 10:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_79(8168936) done Nov 05 10:20:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:20:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_49(8168881) WEXITSTATUS 0 Nov 05 10:20:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_49(8168881) done Nov 05 10:20:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_25(8168818) WEXITSTATUS 0 Nov 05 10:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_25(8168818) done Nov 05 10:20:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_88(8168945) WEXITSTATUS 0 Nov 05 10:20:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_88(8168945) done Nov 05 10:20:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_87(8168944) WEXITSTATUS 0 Nov 05 10:20:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_87(8168944) done Nov 05 10:20:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_129(8168986) WEXITSTATUS 0 Nov 05 10:20:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_129(8168986) done Nov 05 10:20:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169692 InitPrio=27496 usec=3759 Nov 05 10:20:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169692 NodeList=r816u03n09 #CPUs=4 Partition=pi_fedorov Nov 05 10:20:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7753(8168856) WEXITSTATUS 1 Nov 05 10:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7753(8168856) done Nov 05 10:20:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7227(8165875) WEXITSTATUS 1 Nov 05 10:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7227(8165875) done Nov 05 10:21:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_121(8168978) WEXITSTATUS 0 Nov 05 10:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_121(8168978) done Nov 05 10:21:06 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3079(8130394) Nov 05 10:21:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_434(8169693) in pi_ohern on r801u32n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_435(8169694) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_436(8169695) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_437(8169696) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_438(8169697) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_439(8169698) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_440(8169699) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_441(8169700) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_442(8169701) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_443(8169702) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_444(8169703) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_445(8169704) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_446(8169705) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_447(8169706) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_448(8169707) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_449(8169708) in pi_ohern on r917u13n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_450(8169709) in pi_ohern on r917u13n03 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_451(8169710) in pi_ohern on r917u13n03 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_452(8169711) in pi_ohern on r917u13n03 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_453(8169712) in pi_ohern on r917u13n03 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_454(8169713) in pi_ohern on r917u13n03 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_455(8169714) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_456(8169715) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_457(8169716) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_458(8169717) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_459(8169718) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_460(8169719) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_461(8169720) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_462(8169721) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_463(8169722) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_464(8169723) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_465(8169724) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_466(8169725) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_467(8169726) in pi_ohern on r917u13n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_468(8169727) in pi_ohern on r917u29n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_469(8169728) in pi_ohern on r917u29n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_470(8169729) in pi_ohern on r917u29n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_471(8169730) in pi_ohern on r917u29n02 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_472(8169731) in pi_ohern on r917u29n03 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_473(8169732) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_474(8169733) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_475(8169734) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_476(8169735) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_477(8169736) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_478(8169737) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_479(8169738) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_480(8169739) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_481(8169740) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_482(8169741) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_483(8169742) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_484(8169743) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_485(8169744) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_486(8169745) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_487(8169746) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_488(8169747) in pi_ohern on r917u29n04 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_489(8169748) in pi_ohern on r815u03n11 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_490(8169749) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_491(8169750) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_492(8169751) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_493(8169752) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_494(8169753) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_495(8169754) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_496(8169755) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_497(8169756) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_498(8169757) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_499(8169758) in pi_ohern on r815u03n12 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_500(8169759) in pi_ohern on r807u22n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_501(8169760) in pi_ohern on r807u22n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_502(8169761) in pi_ohern on r911u32n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_503(8169762) in pi_ohern on r911u32n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_504(8169763) in pi_ohern on r913u32n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_505(8169764) in pi_ohern on r913u35n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_506(8169765) in pi_ohern on r913u35n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_507(8169766) in pi_ohern on r903u12n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_508(8169767) in pi_ohern on r903u12n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_509(8169768) in pi_ohern on r903u12n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_510(8169769) in pi_ohern on r903u12n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_511(8169770) in pi_ohern on r903u24n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_512(8169771) in pi_ohern on r903u24n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_513(8169772) in pi_ohern on r903u26n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_514(8169773) in pi_ohern on r903u26n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_515(8169774) in pi_ohern on r903u26n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_516(8169775) in pi_ohern on r903u26n01 Nov 05 10:21:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_517(8169776) in pi_ohern on r903u26n01 Nov 05 10:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169691 in day on r815u03n[02-03,08],r815u09n[01,04,08],r815u15n08,r815u23n05,r815u29n[02,04],r815u35n01,r816u15n03 Nov 05 10:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3968(8169777) in gpu on r808u14n01 Nov 05 10:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169565 in day on r918u01n03 Nov 05 10:21:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2922(8169778) in day on r816u15n06 Nov 05 10:21:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166587 in gpu on r915u09n01 Nov 05 10:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_50(8168882) WEXITSTATUS 0 Nov 05 10:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_50(8168882) done Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7860(8169779) in scavenge on r801u04n03 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7861(8169780) in scavenge on r802u04n03 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7862(8169781) in scavenge on r915u32n01 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7863(8169782) in scavenge on r815u29n12 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7864(8169783) in scavenge on r816u29n10 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7865(8169784) in scavenge on r805u25n03 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7866(8169785) in scavenge on r909u04n04 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7867(8169786) in scavenge on r909u10n02 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7868(8169787) in scavenge on r909u10n02 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7869(8169788) in scavenge on r918u09n01 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_48(8168880) WEXITSTATUS 0 Nov 05 10:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_48(8168880) done Nov 05 10:21:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7870(8169789) in scavenge on r919u01n04 Nov 05 10:21:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7871(8169790) in scavenge on r904u10n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169630 in scavenge on r907u09n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169631 in scavenge on r907u09n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169632 in scavenge on r802u14n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169633 in scavenge on r802u23n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169634 in scavenge on r802u30n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169635 in scavenge on r801u06n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169636 in scavenge on r801u06n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169637 in scavenge on r801u09n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169638 in scavenge on r801u09n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169639 in scavenge on r801u27n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169640 in scavenge on r911u09n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169641 in scavenge on r911u09n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169642 in scavenge on r911u09n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169643 in scavenge on r911u13n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169644 in scavenge on r911u24n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169645 in scavenge on r911u28n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169646 in scavenge on r911u28n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169647 in scavenge on r912u05n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169648 in scavenge on r815u15n09 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169649 in scavenge on r815u15n10 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169650 in scavenge on r815u23n07 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169651 in scavenge on r815u29n09 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169652 in scavenge on r816u23n11 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169653 in scavenge on r816u29n08 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169654 in scavenge on r816u29n11 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169655 in scavenge on r808u17n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169656 in scavenge on r4513u23n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169657 in scavenge on r807u14n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169658 in scavenge on r906u39n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169659 in scavenge on r907u32n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169660 in scavenge on r919u32n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169661 in scavenge on r805u06n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169662 in scavenge on r805u06n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169663 in scavenge on r805u09n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169664 in scavenge on r805u09n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169665 in scavenge on r805u09n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169666 in scavenge on r805u14n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169667 in scavenge on r805u14n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169668 in scavenge on r805u14n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169669 in scavenge on r805u23n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169670 in scavenge on r805u25n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169671 in scavenge on r805u25n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169672 in scavenge on r805u27n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169673 in scavenge on r805u27n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169674 in scavenge on r912u28n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169675 in scavenge on r912u28n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169676 in scavenge on r912u28n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169677 in scavenge on r912u28n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169678 in scavenge on r912u32n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169679 in scavenge on r912u32n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169680 in scavenge on r913u01n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169681 in scavenge on r913u09n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169682 in scavenge on r913u09n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169683 in scavenge on r913u24n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169684 in scavenge on r913u28n01 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169685 in scavenge on r913u28n02 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169686 in scavenge on r913u28n04 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169687 in scavenge on r815u15n03 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169688 in scavenge on r815u23n05 Nov 05 10:21:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169689 in scavenge on r815u29n02 Nov 05 10:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_150(8169067) WEXITSTATUS 0 Nov 05 10:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_150(8169067) done Nov 05 10:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_93(8168950) WEXITSTATUS 0 Nov 05 10:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_93(8168950) done Nov 05 10:21:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7587(8167002) WEXITSTATUS 1 Nov 05 10:21:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7587(8167002) done Nov 05 10:21:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_100(8168957) WEXITSTATUS 0 Nov 05 10:21:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_100(8168957) done Nov 05 10:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_70(8168902) WEXITSTATUS 0 Nov 05 10:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_70(8168902) done Nov 05 10:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_122(8168979) WEXITSTATUS 0 Nov 05 10:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_122(8168979) done Nov 05 10:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7527(8166570) WEXITSTATUS 1 Nov 05 10:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7527(8166570) done Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_73(8168930) WEXITSTATUS 0 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_73(8168930) done Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_72(8168929) WEXITSTATUS 0 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_72(8168929) done Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169791 InitPrio=24842 usec=3586 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169792 InitPrio=24840 usec=3158 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169793 InitPrio=24842 usec=3005 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169794 InitPrio=24840 usec=4141 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169795 InitPrio=24842 usec=2967 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169796 InitPrio=24840 usec=3138 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169797 InitPrio=24842 usec=2429 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169798 InitPrio=24840 usec=2101 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169799 InitPrio=24842 usec=2020 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169800 InitPrio=24840 usec=2291 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169801 InitPrio=24842 usec=1961 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169802 InitPrio=24840 usec=1850 Nov 05 10:21:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169803 InitPrio=24842 usec=3063 Nov 05 10:21:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169804 InitPrio=24840 usec=3377 Nov 05 10:21:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169805 InitPrio=24842 usec=3404 Nov 05 10:21:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169806 InitPrio=24840 usec=3558 Nov 05 10:21:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169807 InitPrio=24842 usec=4305 Nov 05 10:21:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169808 InitPrio=24840 usec=3274 Nov 05 10:21:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_37(8168830) WEXITSTATUS 0 Nov 05 10:21:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_37(8168830) done Nov 05 10:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_119(8168976) WEXITSTATUS 0 Nov 05 10:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_119(8168976) done Nov 05 10:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_124(8168981) WEXITSTATUS 0 Nov 05 10:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_124(8168981) done Nov 05 10:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7368(8166110) WEXITSTATUS 1 Nov 05 10:21:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7368(8166110) done Nov 05 10:21:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169791 NodeList=r905u05n[01-04],r905u09n01 #CPUs=120 Partition=mpi Nov 05 10:21:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_182(8169099) WEXITSTATUS 0 Nov 05 10:21:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_182(8169099) done Nov 05 10:21:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_105(8168962) WEXITSTATUS 0 Nov 05 10:21:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_105(8168962) done Nov 05 10:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_99(8168956) WEXITSTATUS 0 Nov 05 10:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_99(8168956) done Nov 05 10:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_116(8168973) WEXITSTATUS 0 Nov 05 10:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_116(8168973) done Nov 05 10:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_162(8169079) WEXITSTATUS 0 Nov 05 10:21:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_162(8169079) done Nov 05 10:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_106(8168963) WEXITSTATUS 0 Nov 05 10:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_106(8168963) done Nov 05 10:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_65(8168897) WEXITSTATUS 0 Nov 05 10:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_65(8168897) done Nov 05 10:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_179(8169096) WEXITSTATUS 0 Nov 05 10:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_179(8169096) done Nov 05 10:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_113(8168970) WEXITSTATUS 0 Nov 05 10:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_113(8168970) done Nov 05 10:22:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2390(8165602) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:22:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2406(8165637) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:22:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2453(8165743) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:22:05 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2634(8166297) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:22:05 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8167606 had to be killed Nov 05 10:22:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2390(8165602) Nov 05 10:22:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2406(8165637) Nov 05 10:22:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2634(8166297) Nov 05 10:22:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2453(8165743) Nov 05 10:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_101(8168958) WEXITSTATUS 0 Nov 05 10:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_101(8168958) done Nov 05 10:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7116(8165695) WEXITSTATUS 1 Nov 05 10:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7116(8165695) done Nov 05 10:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7424(8166213) WEXITSTATUS 1 Nov 05 10:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7424(8166213) done Nov 05 10:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_177(8169094) WEXITSTATUS 0 Nov 05 10:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_177(8169094) done Nov 05 10:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6623(8164688) WEXITSTATUS 1 Nov 05 10:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6623(8164688) done Nov 05 10:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_181(8169098) WEXITSTATUS 0 Nov 05 10:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_181(8169098) done Nov 05 10:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_126(8168983) WEXITSTATUS 0 Nov 05 10:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_126(8168983) done Nov 05 10:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_142(8169059) WEXITSTATUS 0 Nov 05 10:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_142(8169059) done Nov 05 10:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7681(8168324) WEXITSTATUS 1 Nov 05 10:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7681(8168324) done Nov 05 10:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1959(8163591) WEXITSTATUS 0 Nov 05 10:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1959(8163591) done Nov 05 10:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_131(8168988) WEXITSTATUS 0 Nov 05 10:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_131(8168988) done Nov 05 10:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169791 WEXITSTATUS 1 Nov 05 10:22:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169791 done Nov 05 10:22:28 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u05n01 Nov 05 10:22:28 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u05n04 Nov 05 10:22:28 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u05n02 Nov 05 10:22:28 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u05n03 Nov 05 10:22:28 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u09n01 Nov 05 10:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_130(8168987) WEXITSTATUS 0 Nov 05 10:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_130(8168987) done Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169792 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169793 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169794 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169795 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169796 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169797 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169798 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169799 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169800 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169801 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169802 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169803 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169804 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169805 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169806 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169807 Nov 05 10:22:30 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8169808 Nov 05 10:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1867(8161408) WEXITSTATUS 0 Nov 05 10:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1867(8161408) done Nov 05 10:22:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_118(8168975) WEXITSTATUS 0 Nov 05 10:22:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_118(8168975) done Nov 05 10:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7864(8169783) WEXITSTATUS 1 Nov 05 10:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7864(8169783) done Nov 05 10:22:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8169125 uid 22898 Nov 05 10:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_117(8168974) WEXITSTATUS 0 Nov 05 10:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_117(8168974) done Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_128(8168985) WEXITSTATUS 0 Nov 05 10:22:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_128(8168985) done Nov 05 10:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_146(8169063) WEXITSTATUS 0 Nov 05 10:22:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_146(8169063) done Nov 05 10:22:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_94(8168951) WEXITSTATUS 0 Nov 05 10:22:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_94(8168951) done Nov 05 10:22:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_180(8169097) WEXITSTATUS 0 Nov 05 10:22:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_180(8169097) done Nov 05 10:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1981(8163949) WEXITSTATUS 0 Nov 05 10:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1981(8163949) done Nov 05 10:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_220(8169167) WEXITSTATUS 0 Nov 05 10:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_220(8169167) done Nov 05 10:22:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_165(8169082) WEXITSTATUS 0 Nov 05 10:22:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_165(8169082) done Nov 05 10:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_155(8169072) WEXITSTATUS 0 Nov 05 10:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_155(8169072) done Nov 05 10:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2696(8153083) WEXITSTATUS 0 Nov 05 10:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2696(8153083) done Nov 05 10:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_103(8168960) WEXITSTATUS 0 Nov 05 10:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_103(8168960) done Nov 05 10:23:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7530(8166573) WEXITSTATUS 1 Nov 05 10:23:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7530(8166573) done Nov 05 10:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_76(8168933) WEXITSTATUS 0 Nov 05 10:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_76(8168933) done Nov 05 10:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_71(8168928) WEXITSTATUS 0 Nov 05 10:23:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_71(8168928) done Nov 05 10:23:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169809 InitPrio=34739 usec=1986 Nov 05 10:23:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_60(8168892) WEXITSTATUS 0 Nov 05 10:23:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_60(8168892) done Nov 05 10:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_166(8169083) WEXITSTATUS 0 Nov 05 10:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_166(8169083) done Nov 05 10:23:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_148(8169065) WEXITSTATUS 0 Nov 05 10:23:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_148(8169065) done Nov 05 10:23:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_175(8169092) WEXITSTATUS 0 Nov 05 10:23:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_175(8169092) done Nov 05 10:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_111(8168968) WEXITSTATUS 0 Nov 05 10:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_111(8168968) done Nov 05 10:23:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_149(8169066) WEXITSTATUS 0 Nov 05 10:23:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_149(8169066) done Nov 05 10:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7838(8169555) WEXITSTATUS 1 Nov 05 10:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7838(8169555) done Nov 05 10:23:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_108(8168965) WEXITSTATUS 0 Nov 05 10:23:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_108(8168965) done Nov 05 10:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_221(8169168) WEXITSTATUS 0 Nov 05 10:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_221(8169168) done Nov 05 10:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_176(8169093) WEXITSTATUS 0 Nov 05 10:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_176(8169093) done Nov 05 10:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_110(8168967) WEXITSTATUS 0 Nov 05 10:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_110(8168967) done Nov 05 10:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_77(8168934) WEXITSTATUS 0 Nov 05 10:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_77(8168934) done Nov 05 10:23:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169809 NodeList=r903u01n02 #CPUs=4 Partition=scavenge Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_518(8169810) in pi_ohern on r801u32n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_519(8169811) in pi_ohern on r801u32n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_520(8169812) in pi_ohern on r801u32n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_521(8169813) in pi_ohern on r801u32n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_522(8169814) in pi_ohern on r801u32n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_523(8169815) in pi_ohern on r917u13n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_524(8169816) in pi_ohern on r917u13n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_525(8169817) in pi_ohern on r917u13n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_526(8169818) in pi_ohern on r917u13n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_527(8169819) in pi_ohern on r917u13n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_528(8169820) in pi_ohern on r917u13n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_529(8169821) in pi_ohern on r917u13n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_530(8169822) in pi_ohern on r917u13n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_531(8169823) in pi_ohern on r917u13n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_532(8169824) in pi_ohern on r917u13n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_533(8169825) in pi_ohern on r917u13n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_534(8169826) in pi_ohern on r917u29n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_535(8169827) in pi_ohern on r917u29n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_536(8169828) in pi_ohern on r917u29n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_537(8169829) in pi_ohern on r917u29n02 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_538(8169830) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_539(8169831) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_540(8169832) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_541(8169833) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_542(8169834) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_543(8169835) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_544(8169836) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_545(8169837) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_546(8169838) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_547(8169839) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_548(8169840) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_549(8169841) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_550(8169842) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_551(8169843) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_552(8169844) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_553(8169845) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_554(8169846) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_555(8169847) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_556(8169848) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_557(8169849) in pi_ohern on r917u29n03 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_558(8169850) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_559(8169851) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_560(8169852) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_561(8169853) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_562(8169854) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_563(8169855) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_564(8169856) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_565(8169857) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_566(8169858) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_567(8169859) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_568(8169860) in pi_ohern on r917u29n04 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_569(8169861) in pi_ohern on r815u03n11 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_570(8169862) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_571(8169863) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_572(8169864) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_573(8169865) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_574(8169866) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_575(8169867) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_576(8169868) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_577(8169869) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_578(8169870) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_579(8169871) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_580(8169872) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_581(8169873) in pi_ohern on r815u03n12 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_582(8169874) in pi_ohern on r913u35n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_583(8169875) in pi_ohern on r913u35n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_584(8169876) in pi_ohern on r913u35n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_585(8169877) in pi_ohern on r913u35n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_586(8169878) in pi_ohern on r913u35n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_587(8169879) in pi_ohern on r913u35n01 Nov 05 10:23:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_588(8169880) in pi_ohern on r807u22n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_589(8169881) in pi_ohern on r807u24n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_590(8169882) in pi_ohern on r807u24n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_591(8169883) in pi_ohern on r807u24n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_592(8169884) in pi_ohern on r903u12n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_593(8169885) in pi_ohern on r903u12n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_594(8169886) in pi_ohern on r903u12n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_595(8169887) in pi_ohern on r903u12n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_596(8169888) in pi_ohern on r903u24n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_597(8169889) in pi_ohern on r903u24n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_598(8169890) in pi_ohern on r903u24n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_599(8169891) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_600(8169892) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_601(8169893) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_602(8169894) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_603(8169895) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_604(8169896) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_605(8169897) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_606(8169898) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_607(8169899) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_608(8169900) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_609(8169901) in pi_ohern on r903u26n01 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u05n03 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2923(8169902) in day on r909u13n04 Nov 05 10:23:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2924(8169903) in day on r918u05n03 Nov 05 10:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_140(8169057) WEXITSTATUS 0 Nov 05 10:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_140(8169057) done Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7872(8169904) in scavenge on r802u02n03 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7873(8169905) in scavenge on r802u30n01 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7874(8169906) in scavenge on r815u15n12 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7875(8169907) in scavenge on r815u23n09 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7876(8169908) in scavenge on r816u29n10 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7877(8169909) in scavenge on r816u29n11 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7878(8169910) in scavenge on r807u05n03 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7879(8169911) in scavenge on r909u13n01 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7880(8169912) in scavenge on r914u01n02 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7881(8169913) in scavenge on r914u05n01 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_97(8168954) WEXITSTATUS 0 Nov 05 10:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_97(8168954) done Nov 05 10:24:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2635(8166298) in scavenge on r912u13n01 Nov 05 10:24:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2597(8166234) in scavenge on r912u13n01 Nov 05 10:24:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2368(8165575) in scavenge on r912u13n01 Nov 05 10:24:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2391(8165603) in scavenge on r912u13n01 Nov 05 10:24:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2369(8165576) in scavenge on r912u13n01 Nov 05 10:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_75(8168932) WEXITSTATUS 0 Nov 05 10:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_75(8168932) done Nov 05 10:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1967(8163695) WEXITSTATUS 0 Nov 05 10:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1967(8163695) done Nov 05 10:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_163(8169080) WEXITSTATUS 0 Nov 05 10:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_163(8169080) done Nov 05 10:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_183(8169100) WEXITSTATUS 0 Nov 05 10:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_183(8169100) done Nov 05 10:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_173(8169090) WEXITSTATUS 0 Nov 05 10:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_173(8169090) done Nov 05 10:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_228(8169175) WEXITSTATUS 0 Nov 05 10:24:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_228(8169175) done Nov 05 10:24:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169914 InitPrio=16342 usec=5196 Nov 05 10:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_74(8168931) WEXITSTATUS 0 Nov 05 10:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_74(8168931) done Nov 05 10:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_186(8169133) WEXITSTATUS 0 Nov 05 10:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_186(8169133) done Nov 05 10:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7821(8169458) WEXITSTATUS 1 Nov 05 10:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7821(8169458) done Nov 05 10:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_159(8169076) WEXITSTATUS 0 Nov 05 10:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_159(8169076) done Nov 05 10:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3951(8169189) WEXITSTATUS 0 Nov 05 10:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3951(8169189) done Nov 05 10:24:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_141(8169058) WEXITSTATUS 0 Nov 05 10:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_141(8169058) done Nov 05 10:24:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_195(8169142) WEXITSTATUS 0 Nov 05 10:24:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_195(8169142) done Nov 05 10:24:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_167(8169084) WEXITSTATUS 0 Nov 05 10:24:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_167(8169084) done Nov 05 10:24:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_156(8169073) WEXITSTATUS 0 Nov 05 10:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_156(8169073) done Nov 05 10:24:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_112(8168969) WEXITSTATUS 0 Nov 05 10:24:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_112(8168969) done Nov 05 10:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7841(8169558) WEXITSTATUS 1 Nov 05 10:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7841(8169558) done Nov 05 10:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_200(8169147) WEXITSTATUS 0 Nov 05 10:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_200(8169147) done Nov 05 10:24:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_222(8169169) WEXITSTATUS 0 Nov 05 10:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_222(8169169) done Nov 05 10:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7354(8166070) WEXITSTATUS 1 Nov 05 10:24:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7354(8166070) done Nov 05 10:24:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7860(8169779) WEXITSTATUS 1 Nov 05 10:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7860(8169779) done Nov 05 10:24:35 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2405(8165636) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:24:35 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2429(8165678) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:24:35 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2441(8165715) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:24:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_153(8169070) WEXITSTATUS 0 Nov 05 10:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_153(8169070) done Nov 05 10:24:37 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2405(8165636) Nov 05 10:24:37 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2429(8165678) Nov 05 10:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_189(8169136) WEXITSTATUS 0 Nov 05 10:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_189(8169136) done Nov 05 10:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7431(8166220) WEXITSTATUS 1 Nov 05 10:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7431(8166220) done Nov 05 10:24:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2441(8165715) Nov 05 10:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_47(8168879) WEXITSTATUS 0 Nov 05 10:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_47(8168879) done Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:24:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_237(8169184) WEXITSTATUS 0 Nov 05 10:24:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_237(8169184) done Nov 05 10:24:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160528 WEXITSTATUS 0 Nov 05 10:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160528 done Nov 05 10:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_151(8169068) WEXITSTATUS 0 Nov 05 10:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_151(8169068) done Nov 05 10:24:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169915 InitPrio=63372 usec=3612 Nov 05 10:24:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:24:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169915 NodeList=r806u28n01 #CPUs=1 Partition=bigmem Nov 05 10:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_96(8168953) WEXITSTATUS 0 Nov 05 10:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_96(8168953) done Nov 05 10:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169915 WEXITSTATUS 2 Nov 05 10:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169915 done Nov 05 10:24:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_161(8169078) WEXITSTATUS 0 Nov 05 10:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_161(8169078) done Nov 05 10:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_144(8169061) WEXITSTATUS 0 Nov 05 10:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_144(8169061) done Nov 05 10:25:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159211 WEXITSTATUS 0 Nov 05 10:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159211 done Nov 05 10:25:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_152(8169069) WEXITSTATUS 0 Nov 05 10:25:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_152(8169069) done Nov 05 10:25:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_168(8169085) WEXITSTATUS 0 Nov 05 10:25:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_168(8169085) done Nov 05 10:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_107(8168964) WEXITSTATUS 0 Nov 05 10:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_107(8168964) done Nov 05 10:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_238(8169185) WEXITSTATUS 0 Nov 05 10:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_238(8169185) done Nov 05 10:25:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:07 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_178(8169095) WEXITSTATUS 0 Nov 05 10:25:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_178(8169095) done Nov 05 10:25:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7473(8166341) WEXITSTATUS 1 Nov 05 10:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7473(8166341) done Nov 05 10:25:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169916 InitPrio=34788 usec=6696 Nov 05 10:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_98(8168955) WEXITSTATUS 0 Nov 05 10:25:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_98(8168955) done Nov 05 10:25:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_102(8168959) WEXITSTATUS 0 Nov 05 10:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_102(8168959) done Nov 05 10:25:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_209(8169156) WEXITSTATUS 0 Nov 05 10:25:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_209(8169156) done Nov 05 10:25:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_147(8169064) WEXITSTATUS 0 Nov 05 10:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_147(8169064) done Nov 05 10:25:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_232(8169179) WEXITSTATUS 0 Nov 05 10:25:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_232(8169179) done Nov 05 10:25:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169809 WEXITSTATUS 0 Nov 05 10:25:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169809 done Nov 05 10:25:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_115(8168972) WEXITSTATUS 0 Nov 05 10:25:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_115(8168972) done Nov 05 10:25:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:34 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8169917 NodeList=(null) usec=30009 Nov 05 10:25:37 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169917 NodeList=r806u23n04 #CPUs=4 Partition=devel Nov 05 10:25:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3899(8145192) WEXITSTATUS 1 Nov 05 10:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3899(8145192) done Nov 05 10:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_184(8169101) WEXITSTATUS 0 Nov 05 10:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_184(8169101) done Nov 05 10:25:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_219(8169166) WEXITSTATUS 0 Nov 05 10:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_219(8169166) done Nov 05 10:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_207(8169154) WEXITSTATUS 0 Nov 05 10:25:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_207(8169154) done Nov 05 10:25:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_226(8169173) WEXITSTATUS 0 Nov 05 10:25:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_226(8169173) done Nov 05 10:25:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_160(8169077) WEXITSTATUS 0 Nov 05 10:25:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_160(8169077) done Nov 05 10:25:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169918 InitPrio=34706 usec=7520 Nov 05 10:25:49 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u09n01.grace.ycrc.yale.edu" Nov 05 10:25:49 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u09n01.grace.ycrc.yale.edu" Nov 05 10:25:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_215(8169162) WEXITSTATUS 0 Nov 05 10:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_215(8169162) done Nov 05 10:25:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:54 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u05n03.grace.ycrc.yale.edu" Nov 05 10:25:54 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u05n03.grace.ycrc.yale.edu" Nov 05 10:25:58 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u05n02.grace.ycrc.yale.edu" Nov 05 10:25:58 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u05n02.grace.ycrc.yale.edu" Nov 05 10:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_81(8168938) WEXITSTATUS 0 Nov 05 10:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_81(8168938) done Nov 05 10:25:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:25:59 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u05n04.grace.ycrc.yale.edu" Nov 05 10:25:59 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u05n04.grace.ycrc.yale.edu" Nov 05 10:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_211(8169158) WEXITSTATUS 0 Nov 05 10:26:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_211(8169158) done Nov 05 10:26:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7671(8168307) WEXITSTATUS 1 Nov 05 10:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7671(8168307) done Nov 05 10:26:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3965(8169545) WEXITSTATUS 0 Nov 05 10:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3965(8169545) done Nov 05 10:26:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7653(8167258) WEXITSTATUS 1 Nov 05 10:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7653(8167258) done Nov 05 10:26:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169919 InitPrio=16342 usec=3917 Nov 05 10:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_234(8169181) WEXITSTATUS 0 Nov 05 10:26:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_234(8169181) done Nov 05 10:26:12 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u05n01.grace.ycrc.yale.edu" Nov 05 10:26:12 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u05n01.grace.ycrc.yale.edu" Nov 05 10:26:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_231(8169178) WEXITSTATUS 0 Nov 05 10:26:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_231(8169178) done Nov 05 10:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6805(8164975) WEXITSTATUS 1 Nov 05 10:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6805(8164975) done Nov 05 10:26:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_174(8169091) WEXITSTATUS 0 Nov 05 10:26:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_174(8169091) done Nov 05 10:26:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169920 InitPrio=48210 usec=7018 Nov 05 10:26:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_265(8169251) WEXITSTATUS 0 Nov 05 10:26:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_265(8169251) done Nov 05 10:26:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_196(8169143) WEXITSTATUS 0 Nov 05 10:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_196(8169143) done Nov 05 10:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_206(8169153) WEXITSTATUS 0 Nov 05 10:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_206(8169153) done Nov 05 10:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_229(8169176) WEXITSTATUS 0 Nov 05 10:26:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_229(8169176) done Nov 05 10:26:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8169921 InitPrio=34706 usec=3695 Nov 05 10:26:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_255(8169241) WEXITSTATUS 0 Nov 05 10:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_255(8169241) done Nov 05 10:26:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1990(8164089) WEXITSTATUS 0 Nov 05 10:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1990(8164089) done Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_610(8169922) in pi_ohern on r801u32n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_611(8169923) in pi_ohern on r801u32n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_612(8169924) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_613(8169925) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_614(8169926) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_615(8169927) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_616(8169928) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_617(8169929) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_618(8169930) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_619(8169931) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_620(8169932) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_621(8169933) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_622(8169934) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_623(8169935) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_624(8169936) in pi_ohern on r917u13n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_625(8169937) in pi_ohern on r917u13n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_626(8169938) in pi_ohern on r917u13n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_627(8169939) in pi_ohern on r917u13n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_628(8169940) in pi_ohern on r917u13n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_629(8169941) in pi_ohern on r917u13n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_630(8169942) in pi_ohern on r917u13n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_631(8169943) in pi_ohern on r917u13n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_632(8169944) in pi_ohern on r917u13n03 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_633(8169945) in pi_ohern on r917u29n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_634(8169946) in pi_ohern on r917u29n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_635(8169947) in pi_ohern on r917u29n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_636(8169948) in pi_ohern on r917u29n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_637(8169949) in pi_ohern on r917u29n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_638(8169950) in pi_ohern on r917u29n02 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_639(8169951) in pi_ohern on r917u29n03 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_640(8169952) in pi_ohern on r917u29n03 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_641(8169953) in pi_ohern on r917u29n03 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_642(8169954) in pi_ohern on r917u29n03 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_643(8169955) in pi_ohern on r917u29n03 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_644(8169956) in pi_ohern on r917u29n03 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_645(8169957) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_646(8169958) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_647(8169959) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_648(8169960) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_649(8169961) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_650(8169962) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_651(8169963) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_652(8169964) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_653(8169965) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_654(8169966) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_655(8169967) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_656(8169968) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_657(8169969) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_658(8169970) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_659(8169971) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_660(8169972) in pi_ohern on r917u29n04 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_661(8169973) in pi_ohern on r911u32n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_662(8169974) in pi_ohern on r807u22n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_663(8169975) in pi_ohern on r807u22n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_664(8169976) in pi_ohern on r807u22n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_665(8169977) in pi_ohern on r807u22n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_666(8169978) in pi_ohern on r807u24n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_667(8169979) in pi_ohern on r807u24n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168730_668(8168730) in pi_ohern on r807u24n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_1(8169980) in pi_ohern on r903u12n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_2(8169981) in pi_ohern on r903u12n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_3(8169982) in pi_ohern on r903u12n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_4(8169983) in pi_ohern on r903u12n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_5(8169984) in pi_ohern on r903u26n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_6(8169985) in pi_ohern on r903u26n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_7(8169986) in pi_ohern on r903u26n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_8(8169987) in pi_ohern on r913u32n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_9(8169988) in pi_ohern on r913u32n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_10(8169989) in pi_ohern on r913u32n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_11(8169990) in pi_ohern on r913u32n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_12(8169991) in pi_ohern on r913u32n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_13(8169992) in pi_ohern on r913u35n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_14(8169993) in pi_ohern on r913u35n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_15(8169994) in pi_ohern on r913u35n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_16(8169995) in pi_ohern on r913u35n01 Nov 05 10:26:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_17(8169996) in pi_ohern on r913u35n01 Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169920 in gpu_devel on r805u30n01 Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:164 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_187(8169134) WEXITSTATUS 0 Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_187(8169134) done Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_199(8169146) WEXITSTATUS 0 Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_199(8169146) done Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3969(8169997) in gpu on r808u08n01 Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3970(8169998) in gpu on r808u14n01 Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169566 in day on r919u01n01 Nov 05 10:26:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2925(8169999) in day on r909u10n01 Nov 05 10:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_170(8169087) WEXITSTATUS 0 Nov 05 10:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_170(8169087) done Nov 05 10:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_223(8169170) WEXITSTATUS 0 Nov 05 10:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_223(8169170) done Nov 05 10:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_253(8169239) WEXITSTATUS 0 Nov 05 10:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_253(8169239) done Nov 05 10:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_139(8169056) WEXITSTATUS 0 Nov 05 10:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_139(8169056) done Nov 05 10:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6573(8164610) WEXITSTATUS 1 Nov 05 10:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6573(8164610) done Nov 05 10:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_218(8169165) WEXITSTATUS 0 Nov 05 10:26:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_218(8169165) done Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7882(8170000) in scavenge on r801u30n03 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7883(8170001) in scavenge on r801u04n03 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7884(8170002) in scavenge on r903u04n03 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7885(8170003) in scavenge on r805u27n03 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7886(8170004) in scavenge on r807u05n04 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7887(8170005) in scavenge on r914u05n02 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7888(8170006) in scavenge on r914u05n04 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7889(8170007) in scavenge on r912u28n01 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7890(8170008) in scavenge on r913u09n03 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7891(8170009) in scavenge on r913u24n04 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7892(8170010) in scavenge on r815u29n03 Nov 05 10:26:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_202(7571166) in scavenge on r801u25n01 Nov 05 10:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_143(8169060) WEXITSTATUS 0 Nov 05 10:26:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_143(8169060) done Nov 05 10:26:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170011 InitPrio=17468 usec=8078 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:26:48 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170011 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 10:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7823(8169460) WEXITSTATUS 1 Nov 05 10:26:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7823(8169460) done Nov 05 10:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_252(8169238) WEXITSTATUS 0 Nov 05 10:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_252(8169238) done Nov 05 10:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_239(8169186) WEXITSTATUS 0 Nov 05 10:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_239(8169186) done Nov 05 10:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_185(8169132) WEXITSTATUS 0 Nov 05 10:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_185(8169132) done Nov 05 10:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_213(8169160) WEXITSTATUS 0 Nov 05 10:27:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_213(8169160) done Nov 05 10:27:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2450(8165733) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:27:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7137(8165729) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:27:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7158(8165769) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:27:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7505(8166497) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:27:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7679(8168322) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:27:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7158(8165769) Nov 05 10:27:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7137(8165729) Nov 05 10:27:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7679(8168322) Nov 05 10:27:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2450(8165733) Nov 05 10:27:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7505(8166497) Nov 05 10:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_198(8169145) WEXITSTATUS 0 Nov 05 10:27:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_198(8169145) done Nov 05 10:27:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_205(8169152) WEXITSTATUS 0 Nov 05 10:27:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_205(8169152) done Nov 05 10:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_225(8169172) WEXITSTATUS 0 Nov 05 10:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_225(8169172) done Nov 05 10:27:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_217(8169164) WEXITSTATUS 0 Nov 05 10:27:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_217(8169164) done Nov 05 10:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_214(8169161) WEXITSTATUS 0 Nov 05 10:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_214(8169161) done Nov 05 10:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_208(8169155) WEXITSTATUS 0 Nov 05 10:27:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_208(8169155) done Nov 05 10:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_132(8169049) WEXITSTATUS 0 Nov 05 10:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_132(8169049) done Nov 05 10:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_245(8169231) WEXITSTATUS 0 Nov 05 10:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_245(8169231) done Nov 05 10:27:34 r912u23n01 slurmctld[163760]: slurmctld: node r905u09n01 returned to service Nov 05 10:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166662 WEXITSTATUS 0 Nov 05 10:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166662 done Nov 05 10:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_212(8169159) WEXITSTATUS 0 Nov 05 10:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_212(8169159) done Nov 05 10:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_224(8169171) WEXITSTATUS 0 Nov 05 10:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_224(8169171) done Nov 05 10:27:39 r912u23n01 slurmctld[163760]: slurmctld: node r905u05n03 returned to service Nov 05 10:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_169(8169086) WEXITSTATUS 0 Nov 05 10:27:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_169(8169086) done Nov 05 10:27:43 r912u23n01 slurmctld[163760]: slurmctld: node r905u05n02 returned to service Nov 05 10:27:44 r912u23n01 slurmctld[163760]: slurmctld: node r905u05n04 returned to service Nov 05 10:27:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_227(8169174) WEXITSTATUS 0 Nov 05 10:27:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_227(8169174) done Nov 05 10:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_194(8169141) WEXITSTATUS 0 Nov 05 10:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_194(8169141) done Nov 05 10:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_134(8169051) WEXITSTATUS 0 Nov 05 10:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_134(8169051) done Nov 05 10:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_136(8169053) WEXITSTATUS 0 Nov 05 10:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_136(8169053) done Nov 05 10:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_230(8169177) WEXITSTATUS 0 Nov 05 10:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_230(8169177) done Nov 05 10:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_192(8169139) WEXITSTATUS 0 Nov 05 10:27:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_192(8169139) done Nov 05 10:27:57 r912u23n01 slurmctld[163760]: slurmctld: node r905u05n01 returned to service Nov 05 10:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_138(8169055) WEXITSTATUS 0 Nov 05 10:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_138(8169055) done Nov 05 10:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_190(8169137) WEXITSTATUS 0 Nov 05 10:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_190(8169137) done Nov 05 10:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_204(8169151) WEXITSTATUS 0 Nov 05 10:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_204(8169151) done Nov 05 10:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3969(8169997) WEXITSTATUS 0 Nov 05 10:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3969(8169997) done Nov 05 10:28:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_197(8169144) WEXITSTATUS 0 Nov 05 10:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_197(8169144) done Nov 05 10:28:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6726(8164841) WEXITSTATUS 1 Nov 05 10:28:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6726(8164841) done Nov 05 10:28:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_78(8168935) WEXITSTATUS 0 Nov 05 10:28:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_78(8168935) done Nov 05 10:28:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_154(8169071) WEXITSTATUS 0 Nov 05 10:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_154(8169071) done Nov 05 10:28:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_233(8169180) WEXITSTATUS 0 Nov 05 10:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_233(8169180) done Nov 05 10:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_250(8169236) WEXITSTATUS 0 Nov 05 10:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_250(8169236) done Nov 05 10:28:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7852(8169621) WEXITSTATUS 1 Nov 05 10:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7852(8169621) done Nov 05 10:28:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3970(8169998) WEXITSTATUS 0 Nov 05 10:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3970(8169998) done Nov 05 10:28:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_188(8169135) WEXITSTATUS 0 Nov 05 10:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_188(8169135) done Nov 05 10:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_241(8169227) WEXITSTATUS 0 Nov 05 10:28:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_241(8169227) done Nov 05 10:28:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_254(8169240) WEXITSTATUS 0 Nov 05 10:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_254(8169240) done Nov 05 10:28:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_201(8169148) WEXITSTATUS 0 Nov 05 10:28:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_201(8169148) done Nov 05 10:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_261(8169247) WEXITSTATUS 0 Nov 05 10:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_261(8169247) done Nov 05 10:28:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_135(8169052) WEXITSTATUS 0 Nov 05 10:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_135(8169052) done Nov 05 10:28:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7159(8165770) WEXITSTATUS 1 Nov 05 10:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7159(8165770) done Nov 05 10:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_236(8169183) WEXITSTATUS 0 Nov 05 10:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_236(8169183) done Nov 05 10:28:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:28:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_157(8169074) WEXITSTATUS 0 Nov 05 10:28:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_157(8169074) done Nov 05 10:29:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_216(8169163) WEXITSTATUS 0 Nov 05 10:29:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_216(8169163) done Nov 05 10:29:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_280(8169331) WEXITSTATUS 0 Nov 05 10:29:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_280(8169331) done Nov 05 10:29:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3092(8130753) Nov 05 10:29:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_171(8169088) WEXITSTATUS 0 Nov 05 10:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_171(8169088) done Nov 05 10:29:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_263(8169249) WEXITSTATUS 0 Nov 05 10:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_263(8169249) done Nov 05 10:29:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_18(8170012) in pi_ohern on r801u32n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_19(8170013) in pi_ohern on r801u32n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_20(8170014) in pi_ohern on r801u32n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_21(8170015) in pi_ohern on r801u32n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_22(8170016) in pi_ohern on r801u32n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_23(8170017) in pi_ohern on r801u32n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_24(8170018) in pi_ohern on r801u32n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_25(8170019) in pi_ohern on r801u32n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_26(8170020) in pi_ohern on r917u13n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_27(8170021) in pi_ohern on r917u13n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_28(8170022) in pi_ohern on r917u13n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_29(8170023) in pi_ohern on r917u13n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_30(8170024) in pi_ohern on r917u13n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_31(8170025) in pi_ohern on r917u13n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_32(8170026) in pi_ohern on r917u13n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_33(8170027) in pi_ohern on r917u13n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_34(8170028) in pi_ohern on r917u13n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_35(8170029) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_36(8170030) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_37(8170031) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_38(8170032) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_39(8170033) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_40(8170034) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_41(8170035) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_42(8170036) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_43(8170037) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_44(8170038) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_45(8170039) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_46(8170040) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_47(8170041) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_48(8170042) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_49(8170043) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_50(8170044) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_51(8170045) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_52(8170046) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_53(8170047) in pi_ohern on r917u13n03 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_54(8170048) in pi_ohern on r917u29n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_55(8170049) in pi_ohern on r917u29n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_56(8170050) in pi_ohern on r917u29n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_57(8170051) in pi_ohern on r917u29n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_58(8170052) in pi_ohern on r917u29n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_59(8170053) in pi_ohern on r917u29n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_60(8170054) in pi_ohern on r917u29n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_61(8170055) in pi_ohern on r917u29n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_62(8170056) in pi_ohern on r917u29n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_63(8170057) in pi_ohern on r917u29n02 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_64(8170058) in pi_ohern on r917u29n04 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_65(8170059) in pi_ohern on r917u29n04 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_66(8170060) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_67(8170061) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_68(8170062) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_69(8170063) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_70(8170064) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_71(8170065) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_72(8170066) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_73(8170067) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_74(8170068) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_75(8170069) in pi_ohern on r815u03n11 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_76(8170070) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_77(8170071) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_78(8170072) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_79(8170073) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_80(8170074) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_81(8170075) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_82(8170076) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_83(8170077) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_84(8170078) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_85(8170079) in pi_ohern on r815u03n12 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_86(8170080) in pi_ohern on r807u22n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_87(8170081) in pi_ohern on r807u22n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_88(8170082) in pi_ohern on r807u22n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_89(8170083) in pi_ohern on r807u22n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_90(8170084) in pi_ohern on r903u12n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_91(8170085) in pi_ohern on r903u12n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_92(8170086) in pi_ohern on r903u12n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_93(8170087) in pi_ohern on r911u32n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_94(8170088) in pi_ohern on r911u32n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_95(8170089) in pi_ohern on r911u32n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_96(8170090) in pi_ohern on r911u32n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_97(8170091) in pi_ohern on r913u32n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_98(8170092) in pi_ohern on r913u32n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_99(8170093) in pi_ohern on r913u32n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_100(8170094) in pi_ohern on r913u32n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_101(8170095) in pi_ohern on r913u35n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_102(8170096) in pi_ohern on r913u35n01 Nov 05 10:29:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_103(8170097) in pi_ohern on r913u35n01 Nov 05 10:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_172(8169089) WEXITSTATUS 0 Nov 05 10:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_172(8169089) done Nov 05 10:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_264(8169250) WEXITSTATUS 0 Nov 05 10:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_264(8169250) done Nov 05 10:29:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3971(8170098) in gpu on r808u08n01 Nov 05 10:29:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3972(8170099) in gpu on r808u14n01 Nov 05 10:29:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169914 in day on r913u01n03 Nov 05 10:29:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169919 in day on r913u01n03 Nov 05 10:29:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n03 Nov 05 10:29:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133376 WEXITSTATUS 0 Nov 05 10:29:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133376 done Nov 05 10:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7869(8169788) WEXITSTATUS 1 Nov 05 10:29:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7869(8169788) done Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169914 WEXITSTATUS 0 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169914 done Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7893(8170100) in scavenge on r903u07n01 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7894(8170101) in scavenge on r815u15n11 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7895(8170102) in scavenge on r816u15n11 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7896(8170103) in scavenge on r903u40n01 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7897(8170104) in scavenge on r806u06n03 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7898(8170105) in scavenge on r807u08n02 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7899(8170106) in scavenge on r815u03n07 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7900(8170107) in scavenge on r815u09n02 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7901(8170108) in scavenge on r815u09n07 Nov 05 10:29:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7902(8170109) in scavenge on r816u03n01 Nov 05 10:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169919 WEXITSTATUS 0 Nov 05 10:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169919 done Nov 05 10:29:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_202(8169149) WEXITSTATUS 0 Nov 05 10:29:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_202(8169149) done Nov 05 10:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3968(8169777) WEXITSTATUS 0 Nov 05 10:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3968(8169777) done Nov 05 10:29:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154679 WEXITSTATUS 0 Nov 05 10:29:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154679 done Nov 05 10:29:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8168320 Nov 05 10:29:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7008(8165413) WEXITSTATUS 1 Nov 05 10:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7008(8165413) done Nov 05 10:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_251(8169237) WEXITSTATUS 0 Nov 05 10:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_251(8169237) done Nov 05 10:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1958(8163590) WEXITSTATUS 0 Nov 05 10:29:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1958(8163590) done Nov 05 10:29:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_137(8169054) WEXITSTATUS 0 Nov 05 10:29:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_137(8169054) done Nov 05 10:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3972(8170099) WEXITSTATUS 0 Nov 05 10:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3972(8170099) done Nov 05 10:29:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_235(8169182) WEXITSTATUS 0 Nov 05 10:29:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_235(8169182) done Nov 05 10:29:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8170110 NodeList=(null) usec=21997 Nov 05 10:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7593(8167008) WEXITSTATUS 1 Nov 05 10:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7593(8167008) done Nov 05 10:29:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170110 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 10:29:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_164(8169081) WEXITSTATUS 0 Nov 05 10:29:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_164(8169081) done Nov 05 10:29:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_109(8168966) WEXITSTATUS 0 Nov 05 10:30:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_109(8168966) done Nov 05 10:30:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_249(8169235) WEXITSTATUS 0 Nov 05 10:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_249(8169235) done Nov 05 10:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_290(8169364) WEXITSTATUS 0 Nov 05 10:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_290(8169364) done Nov 05 10:30:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7135(8165727) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:30:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7150(8165755) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:30:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2466(8165777) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:30:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2468(8165779) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:30:07 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7595(8167024) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:30:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170111 InitPrio=63372 usec=4392 Nov 05 10:30:08 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7135(8165727) Nov 05 10:30:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7595(8167024) Nov 05 10:30:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7150(8165755) Nov 05 10:30:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_193(8169140) WEXITSTATUS 0 Nov 05 10:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_193(8169140) done Nov 05 10:30:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2468(8165779) Nov 05 10:30:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2466(8165777) Nov 05 10:30:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_243(8169229) WEXITSTATUS 0 Nov 05 10:30:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_243(8169229) done Nov 05 10:30:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_242(8169228) WEXITSTATUS 0 Nov 05 10:30:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_242(8169228) done Nov 05 10:30:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:26 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 20863: QOSMaxSubmitJobPerUserLimit Nov 05 10:30:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_283(8169334) WEXITSTATUS 0 Nov 05 10:30:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_283(8169334) done Nov 05 10:30:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7512(8166516) WEXITSTATUS 1 Nov 05 10:30:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7512(8166516) done Nov 05 10:30:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170112 InitPrio=31343 usec=7936 Nov 05 10:30:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_158(8169075) WEXITSTATUS 0 Nov 05 10:30:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_158(8169075) done Nov 05 10:30:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170113 InitPrio=21718 usec=3144 Nov 05 10:30:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:30:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_303(8169377) WEXITSTATUS 0 Nov 05 10:30:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_303(8169377) done Nov 05 10:30:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_240(8169187) WEXITSTATUS 0 Nov 05 10:30:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_240(8169187) done Nov 05 10:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168561 WEXITSTATUS 0 Nov 05 10:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168561 done Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:30:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170111 NodeList=r806u28n01 #CPUs=1 Partition=bigmem Nov 05 10:30:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r807u05n03 #CPUs=1 Partition=day Nov 05 10:30:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8170112 uid 25151 Nov 05 10:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_258(8169244) WEXITSTATUS 0 Nov 05 10:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_258(8169244) done Nov 05 10:30:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3971(8170098) WEXITSTATUS 0 Nov 05 10:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3971(8170098) done Nov 05 10:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3939(8168839) WEXITSTATUS 0 Nov 05 10:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3939(8168839) done Nov 05 10:30:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_281(8169332) WEXITSTATUS 0 Nov 05 10:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_281(8169332) done Nov 05 10:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7529(8166572) WEXITSTATUS 1 Nov 05 10:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7529(8166572) done Nov 05 10:31:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_203(8169150) WEXITSTATUS 0 Nov 05 10:31:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_203(8169150) done Nov 05 10:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3789(8143792) WEXITSTATUS 1 Nov 05 10:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3789(8143792) done Nov 05 10:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6981(8165327) WEXITSTATUS 1 Nov 05 10:31:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6981(8165327) done Nov 05 10:31:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_256(8169242) WEXITSTATUS 0 Nov 05 10:31:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_256(8169242) done Nov 05 10:31:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_210(8169157) WEXITSTATUS 0 Nov 05 10:31:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_210(8169157) done Nov 05 10:31:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7839(8169556) WEXITSTATUS 1 Nov 05 10:31:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7839(8169556) done Nov 05 10:31:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7899(8170106) WEXITSTATUS 1 Nov 05 10:31:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7899(8170106) done Nov 05 10:31:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_266(8169317) WEXITSTATUS 0 Nov 05 10:31:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_266(8169317) done Nov 05 10:31:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6231(8163412) WEXITSTATUS 1 Nov 05 10:31:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6231(8163412) done Nov 05 10:31:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:31 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170114 InitPrio=39551 usec=7146 Nov 05 10:31:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:33 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:36 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_276(8169327) WEXITSTATUS 0 Nov 05 10:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_276(8169327) done Nov 05 10:31:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1961(8163593) WEXITSTATUS 0 Nov 05 10:31:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1961(8163593) done Nov 05 10:31:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_257(8169243) WEXITSTATUS 0 Nov 05 10:31:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_257(8169243) done Nov 05 10:31:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_306(8169380) WEXITSTATUS 0 Nov 05 10:31:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_306(8169380) done Nov 05 10:31:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_282(8169333) WEXITSTATUS 0 Nov 05 10:31:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_282(8169333) done Nov 05 10:31:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_326(8169400) WEXITSTATUS 0 Nov 05 10:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_326(8169400) done Nov 05 10:31:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170114 NodeList=r806u04n02 #CPUs=2 Partition=day Nov 05 10:31:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_191(8169138) WEXITSTATUS 0 Nov 05 10:31:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_191(8169138) done Nov 05 10:31:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:31:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_260(8169246) WEXITSTATUS 0 Nov 05 10:31:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_260(8169246) done Nov 05 10:31:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_104(8170115) in pi_ohern on r801u32n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_105(8170116) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_106(8170117) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_107(8170118) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_108(8170119) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_109(8170120) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_110(8170121) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_111(8170122) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_112(8170123) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_113(8170124) in pi_ohern on r917u13n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_114(8170125) in pi_ohern on r917u13n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_115(8170126) in pi_ohern on r917u13n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_116(8170127) in pi_ohern on r917u13n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_117(8170128) in pi_ohern on r917u13n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_118(8170129) in pi_ohern on r917u13n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_119(8170130) in pi_ohern on r917u13n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_120(8170131) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_121(8170132) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_122(8170133) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_123(8170134) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_124(8170135) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_125(8170136) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_126(8170137) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_127(8170138) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_128(8170139) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_129(8170140) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_130(8170141) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_131(8170142) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_132(8170143) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_133(8170144) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_134(8170145) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_135(8170146) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_136(8170147) in pi_ohern on r917u29n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_137(8170148) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_138(8170149) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_139(8170150) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_140(8170151) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_141(8170152) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_142(8170153) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_143(8170154) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_144(8170155) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_145(8170156) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_146(8170157) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_147(8170158) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_148(8170159) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_149(8170160) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_150(8170161) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_151(8170162) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_152(8170163) in pi_ohern on r917u29n02 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_153(8170164) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_154(8170165) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_155(8170166) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_156(8170167) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_157(8170168) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_158(8170169) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_159(8170170) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_160(8170171) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_161(8170172) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_162(8170173) in pi_ohern on r917u29n03 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_163(8170174) in pi_ohern on r917u29n04 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_164(8170175) in pi_ohern on r815u03n11 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_165(8170176) in pi_ohern on r815u03n12 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_166(8170177) in pi_ohern on r911u32n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_167(8170178) in pi_ohern on r911u32n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_168(8170179) in pi_ohern on r911u32n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_169(8170180) in pi_ohern on r807u22n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_170(8170181) in pi_ohern on r807u22n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_171(8170182) in pi_ohern on r807u22n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_172(8170183) in pi_ohern on r807u22n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_173(8170184) in pi_ohern on r807u22n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_174(8170185) in pi_ohern on r807u24n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_175(8170186) in pi_ohern on r903u12n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_176(8170187) in pi_ohern on r903u24n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_177(8170188) in pi_ohern on r913u32n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_178(8170189) in pi_ohern on r913u32n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_179(8170190) in pi_ohern on r913u32n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_180(8170191) in pi_ohern on r913u32n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_181(8170192) in pi_ohern on r913u35n01 Nov 05 10:32:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_182(8170193) in pi_ohern on r913u35n01 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7357650 in day on r807u05n03 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170113 in day on r805u04n02 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3973(8170194) in gpu on r808u08n01 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3974(8170195) in gpu on r808u14n01 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3975(8170196) in gpu on r808u14n01 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3976(8170197) in gpu on r808u17n01 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2926(8170198) in day on r913u01n03 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2927(8170199) in day on r918u36n02 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2928(8170200) in day on r909u04n02 Nov 05 10:32:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2929(8170201) in day on r909u29n04 Nov 05 10:32:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166588 in gpu on r915u13n01 Nov 05 10:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_305(8169379) WEXITSTATUS 0 Nov 05 10:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_305(8169379) done Nov 05 10:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_272(8169323) WEXITSTATUS 0 Nov 05 10:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_272(8169323) done Nov 05 10:32:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1984(8163994) WEXITSTATUS 0 Nov 05 10:32:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1984(8163994) done Nov 05 10:32:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_537(7571502) Nov 05 10:32:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_848(7571814) Nov 05 10:32:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_748(7572917) Nov 05 10:32:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_828(7572997) Nov 05 10:32:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_922(7573091) Nov 05 10:32:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_951(7573120) Nov 05 10:32:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_954(7573123) Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7903(8170202) in scavenge on r903u07n01 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7904(8170203) in scavenge on r816u35n11 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7905(8170204) in scavenge on r4513u20n01 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7906(8170205) in scavenge on r913u01n03 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7907(8170206) in scavenge on r913u01n03 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7908(8170207) in scavenge on r918u09n01 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7909(8170208) in scavenge on r909u01n03 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7910(8170209) in scavenge on r909u29n04 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7911(8170210) in scavenge on r909u29n04 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7912(8170211) in scavenge on r815u03n07 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7913(8170212) in scavenge on r815u15n04 Nov 05 10:32:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7914(8170213) in scavenge on r816u03n03 Nov 05 10:32:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3974(8170195) WEXITSTATUS 0 Nov 05 10:32:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3974(8170195) done Nov 05 10:32:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_247(8169233) WEXITSTATUS 0 Nov 05 10:32:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_247(8169233) done Nov 05 10:32:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170214 InitPrio=24842 usec=3776 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170215 InitPrio=24840 usec=4597 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170216 InitPrio=24842 usec=4694 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170217 InitPrio=24840 usec=3177 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170218 InitPrio=24842 usec=2464 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170219 InitPrio=24840 usec=2335 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170220 InitPrio=24842 usec=2324 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170221 InitPrio=24840 usec=2164 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170222 InitPrio=24842 usec=2100 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170223 InitPrio=24840 usec=2013 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170224 InitPrio=24842 usec=1994 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170225 InitPrio=24840 usec=3840 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170226 InitPrio=24842 usec=3163 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170227 InitPrio=24840 usec=2302 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170228 InitPrio=24842 usec=2932 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170229 InitPrio=24840 usec=2569 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170230 InitPrio=24842 usec=1874 Nov 05 10:32:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170231 InitPrio=24840 usec=1868 Nov 05 10:32:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_274(8169325) WEXITSTATUS 0 Nov 05 10:32:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_274(8169325) done Nov 05 10:32:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_279(8169330) WEXITSTATUS 0 Nov 05 10:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_279(8169330) done Nov 05 10:32:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3967(8169614) WEXITSTATUS 0 Nov 05 10:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3967(8169614) done Nov 05 10:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 10:32:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 10:32:36 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 10:32:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2461(8165762) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:32:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2465(8165776) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:32:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2467(8165778) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:32:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2469(8165780) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:32:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2683(8166484) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170232 InitPrio=34739 usec=1919 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170233 InitPrio=34739 usec=1835 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170234 InitPrio=34739 usec=1509 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170235 InitPrio=34739 usec=1391 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170236 InitPrio=34739 usec=1277 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170237 InitPrio=34739 usec=1252 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170238 InitPrio=34739 usec=1113 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170239 InitPrio=34739 usec=1116 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170240 InitPrio=34739 usec=2207 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170241 InitPrio=34739 usec=1263 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170242 InitPrio=34739 usec=1173 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170243 InitPrio=34739 usec=1074 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170244 InitPrio=34739 usec=1060 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170245 InitPrio=34739 usec=1050 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170246 InitPrio=34739 usec=1053 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170247 InitPrio=34739 usec=1059 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170248 InitPrio=34739 usec=1057 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170249 InitPrio=34739 usec=1054 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170250 InitPrio=34739 usec=1107 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170251 InitPrio=34739 usec=1112 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7586(8167001) WEXITSTATUS 1 Nov 05 10:32:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7586(8167001) done Nov 05 10:32:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2683(8166484) Nov 05 10:32:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2469(8165780) Nov 05 10:32:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2467(8165778) Nov 05 10:32:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2461(8165762) Nov 05 10:32:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2465(8165776) Nov 05 10:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_320(8169394) WEXITSTATUS 0 Nov 05 10:32:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_320(8169394) done Nov 05 10:32:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_269(8169320) WEXITSTATUS 0 Nov 05 10:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_269(8169320) done Nov 05 10:32:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_248(8169234) WEXITSTATUS 0 Nov 05 10:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_248(8169234) done Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:32:47 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:32:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170214 NodeList=r905u05n[01-04],r905u09n01 #CPUs=120 Partition=mpi Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170232 NodeList=r802u23n04 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170233 NodeList=r802u25n04 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170234 NodeList=r903u01n02 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170235 NodeList=r903u01n03 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170236 NodeList=r801u04n02 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170237 NodeList=r801u04n04 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170238 NodeList=r801u14n02 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170239 NodeList=r911u01n01 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170240 NodeList=r912u01n04 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170241 NodeList=r914u09n03 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170242 NodeList=r914u09n03 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170243 NodeList=r914u09n03 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170244 NodeList=r914u09n04 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170245 NodeList=r914u09n04 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170246 NodeList=r914u09n04 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170247 NodeList=r914u13n02 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170248 NodeList=r914u13n02 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170249 NodeList=r914u13n02 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170250 NodeList=r914u13n03 #CPUs=4 Partition=scavenge Nov 05 10:32:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170251 NodeList=r914u13n03 #CPUs=4 Partition=scavenge Nov 05 10:32:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8170214 uid 21079 Nov 05 10:32:55 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u05n02 Nov 05 10:32:55 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u05n04 Nov 05 10:32:55 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u09n01 Nov 05 10:32:55 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u05n03 Nov 05 10:32:56 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u05n01 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170215 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170216 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170217 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170218 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170219 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170220 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170221 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170222 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170223 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170224 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170225 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170226 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170227 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170228 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170229 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170230 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170231 Nov 05 10:32:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_277(8169328) WEXITSTATUS 0 Nov 05 10:33:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_277(8169328) done Nov 05 10:33:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1454(8137179) WEXITSTATUS 0 Nov 05 10:33:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1454(8137179) done Nov 05 10:33:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3097(8130950) Nov 05 10:33:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3110(8130963) Nov 05 10:33:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3121(8130974) Nov 05 10:33:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3140(8130993) Nov 05 10:33:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_278(8169329) WEXITSTATUS 0 Nov 05 10:33:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_278(8169329) done Nov 05 10:33:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170252 InitPrio=31343 usec=5183 Nov 05 10:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_304(8169378) WEXITSTATUS 0 Nov 05 10:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_304(8169378) done Nov 05 10:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_328(8169402) WEXITSTATUS 0 Nov 05 10:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_328(8169402) done Nov 05 10:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_296(8169370) WEXITSTATUS 0 Nov 05 10:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_296(8169370) done Nov 05 10:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_318(8169392) WEXITSTATUS 0 Nov 05 10:33:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_318(8169392) done Nov 05 10:33:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170253 InitPrio=31343 usec=5913 Nov 05 10:33:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170254 InitPrio=31343 usec=4629 Nov 05 10:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_267(8169318) WEXITSTATUS 0 Nov 05 10:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_267(8169318) done Nov 05 10:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_1(8168858) WEXITSTATUS 0 Nov 05 10:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_1(8168858) done Nov 05 10:33:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_292(8169366) WEXITSTATUS 0 Nov 05 10:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_292(8169366) done Nov 05 10:33:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8170255 NodeList=(null) usec=29214 Nov 05 10:33:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_259(8169245) WEXITSTATUS 0 Nov 05 10:33:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_259(8169245) done Nov 05 10:33:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170256 InitPrio=47164 usec=7845 Nov 05 10:33:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170257 InitPrio=47164 usec=4246 Nov 05 10:33:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170258 InitPrio=47164 usec=4062 Nov 05 10:33:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170259 InitPrio=47164 usec=2151 Nov 05 10:33:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170260 InitPrio=47164 usec=3385 Nov 05 10:33:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_262(8169248) WEXITSTATUS 0 Nov 05 10:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_262(8169248) done Nov 05 10:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_313(8169387) WEXITSTATUS 0 Nov 05 10:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_313(8169387) done Nov 05 10:33:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 10:33:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 10:33:27 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 10:33:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8170254 uid 25151 Nov 05 10:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_273(8169324) WEXITSTATUS 0 Nov 05 10:33:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_273(8169324) done Nov 05 10:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_133(8169050) WEXITSTATUS 0 Nov 05 10:33:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_133(8169050) done Nov 05 10:33:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170261 InitPrio=31343 usec=4341 Nov 05 10:33:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170262 InitPrio=31343 usec=3994 Nov 05 10:33:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154711 WEXITSTATUS 0 Nov 05 10:33:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154711 done Nov 05 10:33:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_322(8169396) WEXITSTATUS 0 Nov 05 10:33:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_322(8169396) done Nov 05 10:33:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170263 InitPrio=13783 usec=3215 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170264 InitPrio=24842 usec=4854 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170265 InitPrio=24840 usec=4793 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170266 InitPrio=24842 usec=3462 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170267 InitPrio=24840 usec=4000 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170268 InitPrio=24842 usec=3068 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170269 InitPrio=24840 usec=3454 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170270 InitPrio=24842 usec=2734 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170271 InitPrio=24840 usec=3086 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170272 InitPrio=24842 usec=2097 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170273 InitPrio=24840 usec=2531 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170274 InitPrio=24842 usec=2561 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170275 InitPrio=24840 usec=1913 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170276 InitPrio=24842 usec=2595 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170277 InitPrio=24840 usec=2267 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170278 InitPrio=24842 usec=1893 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170279 InitPrio=24840 usec=2980 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170280 InitPrio=24842 usec=2003 Nov 05 10:33:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170281 InitPrio=24840 usec=2547 Nov 05 10:33:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170282 InitPrio=31343 usec=4341 Nov 05 10:33:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8167267 uid 14024 Nov 05 10:33:43 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:105 msg_type=REQUEST_TERMINATE_JOB,REQUEST_TERMINATE_JOB,REQUEST_TERMINATE_JOB,REQUEST_TERMINATE_JOB,REQUEST_TERMINATE_JOB Nov 05 10:33:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170283 InitPrio=31343 usec=4086 Nov 05 10:33:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170284 InitPrio=13783 usec=3149 Nov 05 10:33:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7590(8167005) WEXITSTATUS 1 Nov 05 10:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7590(8167005) done Nov 05 10:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_312(8169386) WEXITSTATUS 0 Nov 05 10:33:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_312(8169386) done Nov 05 10:33:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_324(8169398) WEXITSTATUS 0 Nov 05 10:33:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_324(8169398) done Nov 05 10:33:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_315(8169389) WEXITSTATUS 0 Nov 05 10:33:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_315(8169389) done Nov 05 10:33:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_325(8169399) WEXITSTATUS 0 Nov 05 10:33:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_325(8169399) done Nov 05 10:33:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170255 NodeList=r805u02n01 #CPUs=2 Partition=day Nov 05 10:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170256_1(8170285) NodeList=r4512u32n01 #CPUs=1 Partition=transfer Nov 05 10:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170256_2(8170286) NodeList=r4512u32n01 #CPUs=1 Partition=transfer Nov 05 10:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170264 NodeList=r804u02n[01-04],r804u04n01 #CPUs=120 Partition=mpi Nov 05 10:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170263 NodeList=r914u36n01 #CPUs=4 Partition=gpu_devel Nov 05 10:33:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170284 NodeList=r915u01n01 #CPUs=4 Partition=gpu_devel Nov 05 10:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_244(8169230) WEXITSTATUS 0 Nov 05 10:33:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_244(8169230) done Nov 05 10:33:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170239 WEXITSTATUS 0 Nov 05 10:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170239 done Nov 05 10:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_271(8169322) WEXITSTATUS 0 Nov 05 10:33:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_271(8169322) done Nov 05 10:33:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_285(8169359) WEXITSTATUS 0 Nov 05 10:34:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_285(8169359) done Nov 05 10:34:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8170264 uid 21079 Nov 05 10:34:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:03 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u02n01 Nov 05 10:34:05 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:2300 Nov 05 10:34:06 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u02n03 Nov 05 10:34:06 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u02n02 Nov 05 10:34:06 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u04n01 Nov 05 10:34:06 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u02n04 Nov 05 10:34:06 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:2300 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170265 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170266 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170267 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170268 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170269 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170270 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170271 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170272 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170273 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170274 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170275 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170276 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170277 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170278 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170279 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170280 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170281 Nov 05 10:34:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3147(8131002) Nov 05 10:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_348(8169441) WEXITSTATUS 0 Nov 05 10:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_348(8169441) done Nov 05 10:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170245 WEXITSTATUS 0 Nov 05 10:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170245 done Nov 05 10:34:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_246(8169232) WEXITSTATUS 0 Nov 05 10:34:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_246(8169232) done Nov 05 10:34:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170238 WEXITSTATUS 0 Nov 05 10:34:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170238 done Nov 05 10:34:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_295(8169369) WEXITSTATUS 0 Nov 05 10:34:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_295(8169369) done Nov 05 10:34:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170240 WEXITSTATUS 0 Nov 05 10:34:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170240 done Nov 05 10:34:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_321(8169395) WEXITSTATUS 0 Nov 05 10:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_321(8169395) done Nov 05 10:34:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_286(8169360) WEXITSTATUS 0 Nov 05 10:34:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_286(8169360) done Nov 05 10:34:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170247 WEXITSTATUS 0 Nov 05 10:34:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170247 done Nov 05 10:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_275(8169326) WEXITSTATUS 0 Nov 05 10:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_275(8169326) done Nov 05 10:34:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170236 WEXITSTATUS 0 Nov 05 10:34:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170236 done Nov 05 10:34:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170232 WEXITSTATUS 0 Nov 05 10:34:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170232 done Nov 05 10:34:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170242 WEXITSTATUS 0 Nov 05 10:34:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170242 done Nov 05 10:34:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_967(7573136) Nov 05 10:34:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_987(7573156) Nov 05 10:34:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1012(7573181) Nov 05 10:34:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_1684(8124387) Nov 05 10:34:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:40 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170287 Nov 05 10:34:40 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_find_preemptable_jobs: JobId=8170287 not pending Nov 05 10:34:40 r912u23n01 slurmctld[163760]: error: slurm_find_preemptable_jobs: JobId=8170287 not pending Nov 05 10:34:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170287 InitPrio=4294967294 usec=761 Nov 05 10:34:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170246 WEXITSTATUS 0 Nov 05 10:34:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170246 done Nov 05 10:34:41 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8170288 NodeList=(null) usec=24268 Nov 05 10:34:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170288 NodeList=r917u10n01 #CPUs=1 Partition=pi_econ_lp Nov 05 10:34:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:34:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170251 WEXITSTATUS 0 Nov 05 10:34:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170251 done Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_183(8170289) in pi_ohern on r801u32n02 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_184(8170290) in pi_ohern on r917u13n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_185(8170291) in pi_ohern on r917u13n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_186(8170292) in pi_ohern on r917u13n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_187(8170293) in pi_ohern on r917u13n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_188(8170294) in pi_ohern on r917u13n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_189(8170295) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_190(8170296) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_191(8170297) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_192(8170298) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_193(8170299) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_194(8170300) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_195(8170301) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_196(8170302) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_197(8170303) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_198(8170304) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_199(8170305) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_200(8170306) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_201(8170307) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_202(8170308) in pi_ohern on r917u13n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_203(8170309) in pi_ohern on r917u29n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_204(8170310) in pi_ohern on r917u29n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_205(8170311) in pi_ohern on r917u29n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_206(8170312) in pi_ohern on r917u29n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_207(8170313) in pi_ohern on r917u29n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_208(8170314) in pi_ohern on r917u29n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_209(8170315) in pi_ohern on r917u29n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_210(8170316) in pi_ohern on r917u29n02 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_211(8170317) in pi_ohern on r917u29n02 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_212(8170318) in pi_ohern on r917u29n02 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_213(8170319) in pi_ohern on r917u29n02 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_214(8170320) in pi_ohern on r917u29n02 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_215(8170321) in pi_ohern on r917u29n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_216(8170322) in pi_ohern on r917u29n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_217(8170323) in pi_ohern on r917u29n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_218(8170324) in pi_ohern on r917u29n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_219(8170325) in pi_ohern on r917u29n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_220(8170326) in pi_ohern on r917u29n03 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_221(8170327) in pi_ohern on r815u03n11 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_222(8170328) in pi_ohern on r815u03n11 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_223(8170329) in pi_ohern on r815u03n12 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_224(8170330) in pi_ohern on r903u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_225(8170331) in pi_ohern on r903u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_226(8170332) in pi_ohern on r903u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_227(8170333) in pi_ohern on r903u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_228(8170334) in pi_ohern on r903u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_229(8170335) in pi_ohern on r903u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_230(8170336) in pi_ohern on r913u32n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_231(8170337) in pi_ohern on r913u32n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_232(8170338) in pi_ohern on r913u32n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_233(8170339) in pi_ohern on r807u22n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_234(8170340) in pi_ohern on r807u22n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_235(8170341) in pi_ohern on r807u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_236(8170342) in pi_ohern on r807u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_237(8170343) in pi_ohern on r807u24n01 Nov 05 10:34:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_238(8170344) in pi_ohern on r807u24n01 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170252 in day on r906u41n03 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170253 in day on r906u41n03 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170261 in day on r906u41n03 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170262 in day on r906u41n03 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170282 in day on r906u41n03 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170283 in day on r906u41n04 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3977(8170345) in gpu on r808u14n01 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3978(8170346) in gpu on r808u14n01 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n02 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7651307_0(7651367) in gpu on r4513u23n01 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2930(8170347) in day on r806u09n03 Nov 05 10:34:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2931(8170348) in day on r918u09n04 Nov 05 10:34:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8170287 uid 21079 Nov 05 10:34:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_302(8169376) WEXITSTATUS 0 Nov 05 10:34:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_302(8169376) done Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170349 InitPrio=24842 usec=2760 Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170350 InitPrio=24840 usec=2181 Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170351 InitPrio=24842 usec=2032 Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170352 InitPrio=24840 usec=4375 Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170353 InitPrio=24842 usec=3677 Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170354 InitPrio=24840 usec=3619 Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170355 InitPrio=24842 usec=2751 Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170356 InitPrio=24840 usec=3340 Nov 05 10:34:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170357 InitPrio=24842 usec=1987 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7915(8170358) in scavenge on r802u09n04 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7916(8170359) in scavenge on r802u14n02 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7917(8170360) in scavenge on r802u32n04 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7918(8170361) in scavenge on r815u29n11 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7919(8170362) in scavenge on r816u09n10 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7920(8170363) in scavenge on r816u15n09 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7921(8170364) in scavenge on r816u15n12 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7922(8170365) in scavenge on r808u11n01 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170366 InitPrio=24840 usec=4307 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170367 InitPrio=24842 usec=3345 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170368 InitPrio=24840 usec=2341 Nov 05 10:34:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170369 InitPrio=24842 usec=3838 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170349 NodeList=r804u04n[02-04],r804u06n[01-02] #CPUs=120 Partition=mpi Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170370 InitPrio=24840 usec=2350 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170371 InitPrio=24842 usec=3332 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170372 InitPrio=24840 usec=2687 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170373 InitPrio=24842 usec=2313 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170374 InitPrio=24840 usec=3512 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_528(7572697) in scavenge on r815u23n08 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1059(8115328) in scavenge on r816u23n07 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_809(7571775) in scavenge on r805u02n04 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_892(7571858) in scavenge on r806u09n03 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1185(7572152) in scavenge on r806u14n02 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_624(7572793) in scavenge on r816u23n02 Nov 05 10:34:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_671(8114940) in scavenge on r816u35n03 Nov 05 10:34:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2390(8165602) in scavenge on r906u13n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170233 WEXITSTATUS 0 Nov 05 10:34:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170233 done Nov 05 10:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154689 WEXITSTATUS 0 Nov 05 10:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154689 done Nov 05 10:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_668(8168730) WEXITSTATUS 0 Nov 05 10:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_668(8168730) done Nov 05 10:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7555(8166752) WEXITSTATUS 1 Nov 05 10:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7555(8166752) done Nov 05 10:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_316(8169390) WEXITSTATUS 0 Nov 05 10:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_316(8169390) done Nov 05 10:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170243 WEXITSTATUS 0 Nov 05 10:34:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170243 done Nov 05 10:35:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_311(8169385) WEXITSTATUS 0 Nov 05 10:35:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_311(8169385) done Nov 05 10:35:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_362(8169496) WEXITSTATUS 0 Nov 05 10:35:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_362(8169496) done Nov 05 10:35:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_345(8169438) WEXITSTATUS 0 Nov 05 10:35:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_345(8169438) done Nov 05 10:35:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170250 WEXITSTATUS 0 Nov 05 10:35:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170250 done Nov 05 10:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170241 WEXITSTATUS 0 Nov 05 10:35:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170241 done Nov 05 10:35:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_291(8169365) WEXITSTATUS 0 Nov 05 10:35:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_291(8169365) done Nov 05 10:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_319(8169393) WEXITSTATUS 0 Nov 05 10:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_319(8169393) done Nov 05 10:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170248 WEXITSTATUS 0 Nov 05 10:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170248 done Nov 05 10:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_284(8169358) WEXITSTATUS 0 Nov 05 10:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_284(8169358) done Nov 05 10:35:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_347(8169440) WEXITSTATUS 0 Nov 05 10:35:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_347(8169440) done Nov 05 10:35:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_346(8169439) WEXITSTATUS 0 Nov 05 10:35:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_346(8169439) done Nov 05 10:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_294(8169368) WEXITSTATUS 0 Nov 05 10:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_294(8169368) done Nov 05 10:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_350(8169484) WEXITSTATUS 0 Nov 05 10:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_350(8169484) done Nov 05 10:35:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_299(8169373) WEXITSTATUS 0 Nov 05 10:35:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_299(8169373) done Nov 05 10:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_317(8169391) WEXITSTATUS 0 Nov 05 10:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_317(8169391) done Nov 05 10:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_72(8135771) WEXITSTATUS 0 Nov 05 10:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_72(8135771) done Nov 05 10:35:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_289(8169363) WEXITSTATUS 0 Nov 05 10:35:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_289(8169363) done Nov 05 10:35:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170237 WEXITSTATUS 0 Nov 05 10:35:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170237 done Nov 05 10:35:37 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7226(8165874) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:35:37 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7364(8166102) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:35:37 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7446(8166275) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:35:37 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7588(8167003) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:35:38 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7364(8166102) Nov 05 10:35:39 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7226(8165874) Nov 05 10:35:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7446(8166275) Nov 05 10:35:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7588(8167003) Nov 05 10:35:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5999(8163004) WEXITSTATUS 1 Nov 05 10:35:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5999(8163004) done Nov 05 10:36:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_363(8169497) WEXITSTATUS 0 Nov 05 10:36:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_363(8169497) done Nov 05 10:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170349 WEXITSTATUS 1 Nov 05 10:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170349 done Nov 05 10:36:02 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u04n02 Nov 05 10:36:02 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u04n03 Nov 05 10:36:02 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n01 Nov 05 10:36:02 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n02 Nov 05 10:36:02 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u04n04 Nov 05 10:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_270(8169321) WEXITSTATUS 0 Nov 05 10:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_270(8169321) done Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170350 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170351 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170352 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170353 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170354 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170355 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170356 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170357 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170366 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170367 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170368 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170369 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170370 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170371 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170372 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170373 Nov 05 10:36:04 r912u23n01 slurmctld[163760]: slurmctld: _kill_dependent: Job dependency can't be satisfied, cancelling JobId=8170374 Nov 05 10:36:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8133968 Nov 05 10:36:07 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8168501 Nov 05 10:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7680(8168323) WEXITSTATUS 1 Nov 05 10:36:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7680(8168323) done Nov 05 10:36:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:36:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_344(8169437) WEXITSTATUS 0 Nov 05 10:36:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_344(8169437) done Nov 05 10:36:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_359(8169493) WEXITSTATUS 0 Nov 05 10:36:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_359(8169493) done Nov 05 10:36:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_308(8169382) WEXITSTATUS 0 Nov 05 10:36:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_308(8169382) done Nov 05 10:36:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170234 WEXITSTATUS 0 Nov 05 10:36:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170234 done Nov 05 10:36:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_323(8169397) WEXITSTATUS 0 Nov 05 10:36:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_323(8169397) done Nov 05 10:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_297(8169371) WEXITSTATUS 0 Nov 05 10:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_297(8169371) done Nov 05 10:36:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170235 WEXITSTATUS 0 Nov 05 10:36:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170235 done Nov 05 10:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_358(8169492) WEXITSTATUS 0 Nov 05 10:36:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_358(8169492) done Nov 05 10:36:25 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u05n04.grace.ycrc.yale.edu" Nov 05 10:36:25 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u05n04.grace.ycrc.yale.edu" Nov 05 10:36:26 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u05n03.grace.ycrc.yale.edu" Nov 05 10:36:26 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u05n03.grace.ycrc.yale.edu" Nov 05 10:36:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160532 WEXITSTATUS 0 Nov 05 10:36:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160532 done Nov 05 10:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_352(8169486) WEXITSTATUS 0 Nov 05 10:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_352(8169486) done Nov 05 10:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_288(8169362) WEXITSTATUS 0 Nov 05 10:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_288(8169362) done Nov 05 10:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_355(8169489) WEXITSTATUS 0 Nov 05 10:36:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_355(8169489) done Nov 05 10:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_371(8169505) WEXITSTATUS 0 Nov 05 10:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_371(8169505) done Nov 05 10:36:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:36:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:36:30 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:36:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_310(8169384) WEXITSTATUS 0 Nov 05 10:36:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_310(8169384) done Nov 05 10:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_307(8169381) WEXITSTATUS 0 Nov 05 10:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_307(8169381) done Nov 05 10:36:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170249 WEXITSTATUS 0 Nov 05 10:36:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170249 done Nov 05 10:36:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_343(8169436) WEXITSTATUS 0 Nov 05 10:36:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_343(8169436) done Nov 05 10:36:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_364(8169498) WEXITSTATUS 0 Nov 05 10:36:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_364(8169498) done Nov 05 10:36:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8133968 error Job/step already completing or completed Nov 05 10:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7502(8166494) WEXITSTATUS 1 Nov 05 10:36:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7502(8166494) done Nov 05 10:36:40 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u05n01.grace.ycrc.yale.edu" Nov 05 10:36:40 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u05n01.grace.ycrc.yale.edu" Nov 05 10:36:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170375 InitPrio=31809 usec=7181 Nov 05 10:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7761(8168915) WEXITSTATUS 1 Nov 05 10:36:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7761(8168915) done Nov 05 10:36:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170375 NodeList=r806u33n01 #CPUs=1 Partition=gpu_devel Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:36:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:36:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_314(8169388) WEXITSTATUS 0 Nov 05 10:36:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_314(8169388) done Nov 05 10:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_353(8169487) WEXITSTATUS 0 Nov 05 10:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_353(8169487) done Nov 05 10:37:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_361(8169495) WEXITSTATUS 0 Nov 05 10:37:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_361(8169495) done Nov 05 10:37:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_369(8169503) WEXITSTATUS 0 Nov 05 10:37:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_369(8169503) done Nov 05 10:37:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_367(8169501) WEXITSTATUS 0 Nov 05 10:37:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_367(8169501) done Nov 05 10:37:16 r912u23n01 slurmctld[163760]: slurmctld: node r905u09n01 returned to service Nov 05 10:37:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_338(8169431) WEXITSTATUS 0 Nov 05 10:37:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_338(8169431) done Nov 05 10:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_366(8169500) WEXITSTATUS 0 Nov 05 10:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_366(8169500) done Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_239(8170376) in pi_ohern on r801u32n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_240(8170377) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_241(8170378) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_242(8170379) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_243(8170380) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_244(8170381) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_245(8170382) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_246(8170383) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_247(8170384) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_248(8170385) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_249(8170386) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_250(8170387) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_251(8170388) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_252(8170389) in pi_ohern on r917u13n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_253(8170390) in pi_ohern on r917u13n03 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_254(8170391) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_255(8170392) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_256(8170393) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_257(8170394) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_258(8170395) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_259(8170396) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_260(8170397) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_261(8170398) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_262(8170399) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_263(8170400) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_264(8170401) in pi_ohern on r917u13n04 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_265(8170402) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_266(8170403) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_267(8170404) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_268(8170405) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_269(8170406) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_270(8170407) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_271(8170408) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_272(8170409) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_273(8170410) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_274(8170411) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_275(8170412) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_276(8170413) in pi_ohern on r917u29n02 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_277(8170414) in pi_ohern on r917u29n03 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_278(8170415) in pi_ohern on r917u29n03 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_279(8170416) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_280(8170417) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_281(8170418) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_282(8170419) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_283(8170420) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_284(8170421) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_285(8170422) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_286(8170423) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_287(8170424) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_288(8170425) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_289(8170426) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_290(8170427) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_291(8170428) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_292(8170429) in pi_ohern on r815u03n11 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_293(8170430) in pi_ohern on r815u03n12 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_294(8170431) in pi_ohern on r815u03n12 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_295(8170432) in pi_ohern on r815u03n12 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_296(8170433) in pi_ohern on r815u03n12 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_297(8170434) in pi_ohern on r815u03n12 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_298(8170435) in pi_ohern on r815u03n12 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_299(8170436) in pi_ohern on r807u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_300(8170437) in pi_ohern on r807u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_301(8170438) in pi_ohern on r807u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_302(8170439) in pi_ohern on r807u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_303(8170440) in pi_ohern on r807u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_304(8170441) in pi_ohern on r807u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_305(8170442) in pi_ohern on r807u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_306(8170443) in pi_ohern on r807u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_307(8170444) in pi_ohern on r903u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_308(8170445) in pi_ohern on r903u24n01 Nov 05 10:37:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2932(8170446) in day on r816u09n07 Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_287(8169361) WEXITSTATUS 0 Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_287(8169361) done Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_365(8169499) WEXITSTATUS 0 Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_365(8169499) done Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: node r905u05n02 returned to service Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_368(8169502) WEXITSTATUS 0 Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_368(8169502) done Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_268(8169319) WEXITSTATUS 0 Nov 05 10:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_268(8169319) done Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7923(8170447) in scavenge on r908u28n02 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7924(8170448) in scavenge on r815u09n12 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7925(8170449) in scavenge on r815u15n09 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7926(8170450) in scavenge on r805u04n02 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7927(8170451) in scavenge on r904u04n02 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7928(8170452) in scavenge on r914u05n01 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7929(8170453) in scavenge on r914u05n01 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7930(8170454) in scavenge on r918u01n01 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7931(8170455) in scavenge on r807u05n02 Nov 05 10:37:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_1(8170456) in scavenge on r807u37n01 Nov 05 10:37:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_872(8115141) in scavenge on r801u14n03 Nov 05 10:37:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_638(8114907) in scavenge on r816u09n07 Nov 05 10:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6902(8165149) WEXITSTATUS 1 Nov 05 10:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6902(8165149) done Nov 05 10:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144138 WEXITSTATUS 0 Nov 05 10:37:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144138 done Nov 05 10:37:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_370(8169504) WEXITSTATUS 0 Nov 05 10:37:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_370(8169504) done Nov 05 10:37:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_341(8169434) WEXITSTATUS 0 Nov 05 10:37:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_341(8169434) done Nov 05 10:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3975(8170196) WEXITSTATUS 0 Nov 05 10:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3975(8170196) done Nov 05 10:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_1(8170456) WEXITSTATUS 1 Nov 05 10:37:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_1(8170456) done Nov 05 10:37:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_293(8169367) WEXITSTATUS 0 Nov 05 10:37:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_293(8169367) done Nov 05 10:37:45 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u02n01.grace.ycrc.yale.edu" Nov 05 10:37:45 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u02n01.grace.ycrc.yale.edu" Nov 05 10:37:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:37:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_381(8169534) WEXITSTATUS 0 Nov 05 10:37:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_381(8169534) done Nov 05 10:37:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:37:48 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u02n04.grace.ycrc.yale.edu" Nov 05 10:37:48 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u02n04.grace.ycrc.yale.edu" Nov 05 10:37:49 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u02n02.grace.ycrc.yale.edu" Nov 05 10:37:49 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u02n02.grace.ycrc.yale.edu" Nov 05 10:37:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:37:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144151 NodeList=r816u09n12,r917u01n01 #CPUs=96 Partition=scavenge Nov 05 10:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_356(8169490) WEXITSTATUS 0 Nov 05 10:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_356(8169490) done Nov 05 10:37:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:37:53 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u04n01.grace.ycrc.yale.edu" Nov 05 10:37:53 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u04n01.grace.ycrc.yale.edu" Nov 05 10:37:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170283 WEXITSTATUS 0 Nov 05 10:37:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170283 done Nov 05 10:37:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166800 uid 20354 Nov 05 10:37:55 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u02n03.grace.ycrc.yale.edu" Nov 05 10:37:55 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u02n03.grace.ycrc.yale.edu" Nov 05 10:37:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_360(8169494) WEXITSTATUS 0 Nov 05 10:38:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_360(8169494) done Nov 05 10:38:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8166579 uid 20354 Nov 05 10:38:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:08 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7628(8167078) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:38:08 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7657(8167609) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_301(8169375) WEXITSTATUS 0 Nov 05 10:38:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_301(8169375) done Nov 05 10:38:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7628(8167078) Nov 05 10:38:10 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7657(8167609) Nov 05 10:38:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:10 r912u23n01 slurmctld[163760]: slurmctld: node r905u05n04 returned to service Nov 05 10:38:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_340(8169433) WEXITSTATUS 0 Nov 05 10:38:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_340(8169433) done Nov 05 10:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1978(8163837) WEXITSTATUS 0 Nov 05 10:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1978(8163837) done Nov 05 10:38:12 r912u23n01 slurmctld[163760]: slurmctld: node r905u05n03 returned to service Nov 05 10:38:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_354(8169488) WEXITSTATUS 0 Nov 05 10:38:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_354(8169488) done Nov 05 10:38:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:25 r912u23n01 slurmctld[163760]: slurmctld: node r905u05n01 returned to service Nov 05 10:38:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_357(8169491) WEXITSTATUS 0 Nov 05 10:38:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_357(8169491) done Nov 05 10:38:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3959(8169403) WEXITSTATUS 0 Nov 05 10:38:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3959(8169403) done Nov 05 10:38:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7481(8166438) WEXITSTATUS 1 Nov 05 10:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7481(8166438) done Nov 05 10:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_339(8169432) WEXITSTATUS 0 Nov 05 10:38:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_339(8169432) done Nov 05 10:38:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_298(8169372) WEXITSTATUS 0 Nov 05 10:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_298(8169372) done Nov 05 10:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1463(8138140) WEXITSTATUS 0 Nov 05 10:38:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1463(8138140) done Nov 05 10:38:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_336(8169429) WEXITSTATUS 0 Nov 05 10:38:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_336(8169429) done Nov 05 10:38:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170457 InitPrio=17468 usec=4234 Nov 05 10:38:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_327(8169401) WEXITSTATUS 0 Nov 05 10:38:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_327(8169401) done Nov 05 10:38:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:38:51 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144149 NodeList=r911u01n03,r911u05n03 #CPUs=72 Partition=scavenge Nov 05 10:38:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_342(8169435) WEXITSTATUS 0 Nov 05 10:38:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_342(8169435) done Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:38:53 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:38:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_309(8169383) WEXITSTATUS 0 Nov 05 10:39:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_309(8169383) done Nov 05 10:39:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6632(8164697) WEXITSTATUS 1 Nov 05 10:39:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6632(8164697) done Nov 05 10:39:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_382(8169535) WEXITSTATUS 0 Nov 05 10:39:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_382(8169535) done Nov 05 10:39:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_330(8169423) WEXITSTATUS 0 Nov 05 10:39:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_330(8169423) done Nov 05 10:39:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_335(8169428) WEXITSTATUS 0 Nov 05 10:39:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_335(8169428) done Nov 05 10:39:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170244 WEXITSTATUS 0 Nov 05 10:39:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170244 done Nov 05 10:39:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170262 WEXITSTATUS 0 Nov 05 10:39:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170262 done Nov 05 10:39:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170458 InitPrio=21315 usec=3265 Nov 05 10:39:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170253 WEXITSTATUS 0 Nov 05 10:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170253 done Nov 05 10:39:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_337(8169430) WEXITSTATUS 0 Nov 05 10:39:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_337(8169430) done Nov 05 10:39:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170282 WEXITSTATUS 0 Nov 05 10:39:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170282 done Nov 05 10:39:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170261 WEXITSTATUS 0 Nov 05 10:39:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170261 done Nov 05 10:39:30 r912u23n01 slurmctld[163760]: slurmctld: node r804u02n01 returned to service Nov 05 10:39:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:33 r912u23n01 slurmctld[163760]: slurmctld: node r804u02n04 returned to service Nov 05 10:39:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_373(8169507) WEXITSTATUS 0 Nov 05 10:39:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_373(8169507) done Nov 05 10:39:34 r912u23n01 slurmctld[163760]: slurmctld: node r804u02n02 returned to service Nov 05 10:39:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_349(8169483) WEXITSTATUS 0 Nov 05 10:39:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_349(8169483) done Nov 05 10:39:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:38 r912u23n01 slurmctld[163760]: slurmctld: node r804u04n01 returned to service Nov 05 10:39:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:40 r912u23n01 slurmctld[163760]: slurmctld: node r804u02n03 returned to service Nov 05 10:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_383(8169536) WEXITSTATUS 0 Nov 05 10:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_383(8169536) done Nov 05 10:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_372(8169506) WEXITSTATUS 0 Nov 05 10:39:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_372(8169506) done Nov 05 10:39:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_384(8169537) WEXITSTATUS 0 Nov 05 10:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_384(8169537) done Nov 05 10:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_300(8169374) WEXITSTATUS 0 Nov 05 10:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_300(8169374) done Nov 05 10:39:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170459 InitPrio=57385 usec=5732 Nov 05 10:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170252 WEXITSTATUS 0 Nov 05 10:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170252 done Nov 05 10:39:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170459 NodeList=r915u38n03 #CPUs=1 Partition=pi_lora Nov 05 10:39:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:46 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n01.grace.ycrc.yale.edu" Nov 05 10:39:46 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n01.grace.ycrc.yale.edu" Nov 05 10:39:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7715(8168528) WEXITSTATUS 1 Nov 05 10:39:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7715(8168528) done Nov 05 10:39:46 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u04n04.grace.ycrc.yale.edu" Nov 05 10:39:46 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u04n04.grace.ycrc.yale.edu" Nov 05 10:39:46 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u04n03.grace.ycrc.yale.edu" Nov 05 10:39:46 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u04n03.grace.ycrc.yale.edu" Nov 05 10:39:48 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n02.grace.ycrc.yale.edu" Nov 05 10:39:48 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n02.grace.ycrc.yale.edu" Nov 05 10:39:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7917(8170360) WEXITSTATUS 1 Nov 05 10:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7917(8170360) done Nov 05 10:39:50 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u04n02.grace.ycrc.yale.edu" Nov 05 10:39:50 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u04n02.grace.ycrc.yale.edu" Nov 05 10:39:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170458 NodeList=r915u05n01 #CPUs=4 Partition=gpu_devel Nov 05 10:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168541 WEXITSTATUS 0 Nov 05 10:39:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168541 done Nov 05 10:39:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170460 InitPrio=10773 usec=7026 Nov 05 10:39:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170460 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 10:39:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:39:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7501(8166483) WEXITSTATUS 1 Nov 05 10:39:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7501(8166483) done Nov 05 10:39:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7634(8167084) WEXITSTATUS 1 Nov 05 10:39:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7634(8167084) done Nov 05 10:40:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_4(8168870) WEXITSTATUS 0 Nov 05 10:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_4(8168870) done Nov 05 10:40:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_351(8169485) WEXITSTATUS 0 Nov 05 10:40:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_351(8169485) done Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_309(8170461) in pi_ohern on r801u32n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_310(8170462) in pi_ohern on r917u13n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_311(8170463) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_312(8170464) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_313(8170465) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_314(8170466) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_315(8170467) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_316(8170468) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_317(8170469) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_318(8170470) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_319(8170471) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_320(8170472) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_321(8170473) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_322(8170474) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_323(8170475) in pi_ohern on r917u13n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_324(8170476) in pi_ohern on r917u13n03 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_325(8170477) in pi_ohern on r917u13n04 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_326(8170478) in pi_ohern on r917u13n04 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_327(8170479) in pi_ohern on r917u13n04 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_328(8170480) in pi_ohern on r917u13n04 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_329(8170481) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_330(8170482) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_331(8170483) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_332(8170484) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_333(8170485) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_334(8170486) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_335(8170487) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_336(8170488) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_337(8170489) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_338(8170490) in pi_ohern on r917u29n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_339(8170491) in pi_ohern on r917u29n02 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_340(8170492) in pi_ohern on r917u29n03 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_341(8170493) in pi_ohern on r917u29n03 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_342(8170494) in pi_ohern on r917u29n03 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_343(8170495) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_344(8170496) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_345(8170497) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_346(8170498) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_347(8170499) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_348(8170500) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_349(8170501) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_350(8170502) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_351(8170503) in pi_ohern on r815u03n11 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_352(8170504) in pi_ohern on r815u03n12 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_353(8170505) in pi_ohern on r815u03n12 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_354(8170506) in pi_ohern on r815u03n12 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_355(8170507) in pi_ohern on r815u03n12 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_356(8170508) in pi_ohern on r815u03n12 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_357(8170509) in pi_ohern on r815u03n12 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_358(8170510) in pi_ohern on r807u24n01 Nov 05 10:40:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_359(8170511) in pi_ohern on r903u24n01 Nov 05 10:40:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169918 in day on r919u32n04 Nov 05 10:40:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3979(8170512) in gpu on r808u08n01 Nov 05 10:40:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3980(8170513) in gpu on r808u14n01 Nov 05 10:40:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 10:40:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2933(8170514) in day on r919u01n03 Nov 05 10:40:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_1099(7572065) Nov 05 10:40:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_349(7572517) Nov 05 10:40:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3162(8131553) Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7932(8170515) in scavenge on r802u32n04 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7933(8170516) in scavenge on r807u05n03 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7934(8170517) in scavenge on r913u01n02 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7935(8170518) in scavenge on r913u01n02 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7936(8170519) in scavenge on r919u09n01 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7937(8170520) in scavenge on r919u32n04 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7938(8170521) in scavenge on r919u32n04 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7939(8170522) in scavenge on r919u32n04 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7940(8170523) in scavenge on r805u06n03 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7941(8170524) in scavenge on r805u09n03 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7942(8170525) in scavenge on r805u14n02 Nov 05 10:40:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_2(8170526) in scavenge on r807u37n01 Nov 05 10:40:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2406(8165637) in scavenge on r816u29n11 Nov 05 10:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7894(8170101) WEXITSTATUS 1 Nov 05 10:40:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7894(8170101) done Nov 05 10:40:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_2(8170526) WEXITSTATUS 1 Nov 05 10:40:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_2(8170526) done Nov 05 10:40:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7773(8169046) WEXITSTATUS 1 Nov 05 10:40:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7773(8169046) done Nov 05 10:40:33 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8170527 NodeList=(null) usec=24125 Nov 05 10:40:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170527 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 10:40:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_7(8168873) WEXITSTATUS 0 Nov 05 10:40:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_7(8168873) done Nov 05 10:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3980(8170513) WEXITSTATUS 0 Nov 05 10:40:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3980(8170513) done Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:40:54 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:40:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8168531 Nov 05 10:41:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3977(8170345) WEXITSTATUS 0 Nov 05 10:41:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3977(8170345) done Nov 05 10:41:09 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:41:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_385(8169538) WEXITSTATUS 0 Nov 05 10:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_385(8169538) done Nov 05 10:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_331(8169424) WEXITSTATUS 0 Nov 05 10:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_331(8169424) done Nov 05 10:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6750(8164881) WEXITSTATUS 1 Nov 05 10:41:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6750(8164881) done Nov 05 10:41:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170528 InitPrio=23568 usec=4081 Nov 05 10:41:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1938(8163398) WEXITSTATUS 0 Nov 05 10:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1938(8163398) done Nov 05 10:41:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7720(8168546) WEXITSTATUS 1 Nov 05 10:41:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7720(8168546) done Nov 05 10:41:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128275 WEXITSTATUS 0 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128275 done Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u04n03 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u04n02 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u04n01 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u02n03 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u02n04 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u02n01 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u06n01 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u06n02 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u02n02 Nov 05 10:41:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u04n04 Nov 05 10:41:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7615(8167060) WEXITSTATUS 1 Nov 05 10:41:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7615(8167060) done Nov 05 10:41:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_329(8169422) WEXITSTATUS 0 Nov 05 10:41:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_329(8169422) done Nov 05 10:41:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:30 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n01 returned to service Nov 05 10:41:31 r912u23n01 slurmctld[163760]: slurmctld: node r804u04n03 returned to service Nov 05 10:41:31 r912u23n01 slurmctld[163760]: slurmctld: node r804u04n04 returned to service Nov 05 10:41:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_333(8169426) WEXITSTATUS 0 Nov 05 10:41:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_333(8169426) done Nov 05 10:41:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:33 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n02 returned to service Nov 05 10:41:35 r912u23n01 slurmctld[163760]: slurmctld: node r804u04n02 returned to service Nov 05 10:41:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7677(8168313) WEXITSTATUS 1 Nov 05 10:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7677(8168313) done Nov 05 10:41:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3663(8162915) WEXITSTATUS 0 Nov 05 10:41:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3663(8162915) done Nov 05 10:41:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7849(8169618) WEXITSTATUS 1 Nov 05 10:41:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7849(8169618) done Nov 05 10:41:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7400(8166156) WEXITSTATUS 1 Nov 05 10:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7400(8166156) done Nov 05 10:41:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_401(8169580) WEXITSTATUS 0 Nov 05 10:41:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_401(8169580) done Nov 05 10:41:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3809(8165058) WEXITSTATUS 0 Nov 05 10:41:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3809(8165058) done Nov 05 10:41:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170529 InitPrio=34304 usec=3750 Nov 05 10:41:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_9(8168866) WEXITSTATUS 0 Nov 05 10:41:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_9(8168866) done Nov 05 10:41:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:41:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170530 InitPrio=34304 usec=4224 Nov 05 10:42:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8170531 NodeList=(null) usec=37952 Nov 05 10:42:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_5(8168862) WEXITSTATUS 0 Nov 05 10:42:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_5(8168862) done Nov 05 10:42:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170531 NodeList=r903u34n01 #CPUs=1 Partition=education_gpu Nov 05 10:42:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159212 WEXITSTATUS 0 Nov 05 10:42:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159212 done Nov 05 10:42:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_2(8168859) WEXITSTATUS 0 Nov 05 10:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_2(8168859) done Nov 05 10:42:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_10(8168729) WEXITSTATUS 0 Nov 05 10:42:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_10(8168729) done Nov 05 10:42:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_386(8169539) WEXITSTATUS 0 Nov 05 10:42:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_386(8169539) done Nov 05 10:42:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_402(8169581) WEXITSTATUS 0 Nov 05 10:42:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_402(8169581) done Nov 05 10:42:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_404(8169583) WEXITSTATUS 0 Nov 05 10:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_404(8169583) done Nov 05 10:42:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_6(8168872) WEXITSTATUS 0 Nov 05 10:42:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_6(8168872) done Nov 05 10:42:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7882(8170000) WEXITSTATUS 1 Nov 05 10:42:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7882(8170000) done Nov 05 10:42:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_4(8168861) WEXITSTATUS 0 Nov 05 10:42:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_4(8168861) done Nov 05 10:42:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_6(8168863) WEXITSTATUS 0 Nov 05 10:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_6(8168863) done Nov 05 10:42:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1045(7573214) Nov 05 10:42:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_8(8168865) WEXITSTATUS 0 Nov 05 10:42:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_8(8168865) done Nov 05 10:42:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_360(8170532) in pi_ohern on r801u32n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_361(8170533) in pi_ohern on r801u32n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_362(8170534) in pi_ohern on r801u32n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_363(8170535) in pi_ohern on r815u03n11 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_364(8170536) in pi_ohern on r815u03n11 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_365(8170537) in pi_ohern on r815u03n11 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_366(8170538) in pi_ohern on r815u03n12 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_367(8170539) in pi_ohern on r903u12n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170529 in day on r904u07n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170530 in day on r904u07n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_0(8170540) in day on r904u07n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_1(8170541) in day on r904u07n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_2(8170542) in day on r904u07n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_3(8170543) in day on r904u07n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_4(8170544) in day on r904u07n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_5(8170545) in day on r906u41n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_6(8170546) in day on r906u41n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_7(8170547) in day on r906u41n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_8(8170548) in day on r906u41n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_9(8170549) in day on r906u41n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_10(8170550) in day on r906u41n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_11(8170551) in day on r906u41n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_12(8170552) in day on r904u04n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_13(8170553) in day on r904u04n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_14(8170554) in day on r904u04n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_15(8170555) in day on r904u04n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_16(8170556) in day on r904u04n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_17(8170557) in day on r904u04n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_18(8170558) in day on r904u04n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_19(8170559) in day on r904u04n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_20(8170560) in day on r904u04n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_21(8170561) in day on r904u04n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_22(8170562) in day on r904u04n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_23(8170563) in day on r904u04n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_24(8170564) in day on r904u04n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_25(8170565) in day on r904u04n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_26(8170566) in day on r904u04n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_27(8170567) in day on r904u04n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_28(8170568) in day on r904u04n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_29(8170569) in day on r904u04n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_30(8170570) in day on r904u04n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_31(8170571) in day on r904u10n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_32(8170572) in day on r904u10n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_33(8170573) in day on r904u10n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_34(8170574) in day on r904u10n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_35(8170575) in day on r904u10n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_36(8170576) in day on r904u10n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_37(8170577) in day on r904u10n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_38(8170578) in day on r904u10n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_39(8170579) in day on r904u10n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_40(8170580) in day on r904u10n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_41(8170581) in day on r904u10n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_42(8170582) in day on r904u10n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_43(8170583) in day on r904u10n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_44(8170584) in day on r904u10n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_45(8170585) in day on r904u10n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_46(8170586) in day on r914u01n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_47(8170587) in day on r914u01n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_48(8170588) in day on r914u01n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_49(8170589) in day on r914u01n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_50(8170590) in day on r914u01n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_51(8170591) in day on r914u01n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_52(8170592) in day on r914u01n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_53(8170593) in day on r914u01n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_54(8170594) in day on r914u01n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_55(8170595) in day on r914u01n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_56(8170596) in day on r914u01n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_57(8170597) in day on r914u01n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_58(8170598) in day on r914u01n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_59(8170599) in day on r914u05n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_60(8170600) in day on r914u05n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_61(8170601) in day on r914u05n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_62(8170602) in day on r914u05n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_63(8170603) in day on r914u05n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_64(8170604) in day on r914u05n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_65(8170605) in day on r914u05n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_66(8170606) in day on r914u05n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_67(8170607) in day on r914u05n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_68(8170608) in day on r807u03n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_69(8170609) in day on r807u03n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_70(8170610) in day on r807u03n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_71(8170611) in day on r807u03n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_72(8170612) in day on r807u03n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_73(8170613) in day on r807u03n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_74(8170614) in day on r807u03n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_75(8170615) in day on r807u03n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_76(8170616) in day on r807u03n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_77(8170617) in day on r807u03n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_78(8170618) in day on r807u03n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_79(8170619) in day on r807u05n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_80(8170620) in day on r807u05n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_81(8170621) in day on r807u05n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_82(8170622) in day on r807u05n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_83(8170623) in day on r807u05n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_84(8170624) in day on r807u05n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_85(8170625) in day on r807u05n02 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_86(8170626) in day on r807u05n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_87(8170627) in day on r807u05n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_88(8170628) in day on r807u05n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_89(8170629) in day on r807u05n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_90(8170630) in day on r807u05n03 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_91(8170631) in day on r807u05n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_92(8170632) in day on r807u05n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_93(8170633) in day on r807u05n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_94(8170634) in day on r807u05n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_95(8170635) in day on r807u05n04 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_96(8170636) in day on r807u08n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_97(8170637) in day on r807u08n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_98(8170638) in day on r807u08n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_99(8170639) in day on r807u08n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3981(8170640) in gpu on r808u14n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3982(8170641) in gpu on r808u14n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3983(8170642) in gpu on r808u14n01 Nov 05 10:42:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3984(8170643) in gpu on r903u34n01 Nov 05 10:42:46 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:230 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 10:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_391(8169544) WEXITSTATUS 0 Nov 05 10:42:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_391(8169544) done Nov 05 10:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6880(8165110) WEXITSTATUS 1 Nov 05 10:42:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6880(8165110) done Nov 05 10:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7640(8167095) WEXITSTATUS 1 Nov 05 10:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7640(8167095) done Nov 05 10:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7871(8169790) WEXITSTATUS 1 Nov 05 10:42:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7871(8169790) done Nov 05 10:42:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_410(8169589) WEXITSTATUS 0 Nov 05 10:42:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_410(8169589) done Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7943(8170644) in scavenge on r801u30n03 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7944(8170645) in scavenge on r907u01n04 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7945(8170646) in scavenge on r802u06n01 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7946(8170647) in scavenge on r903u01n01 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7947(8170648) in scavenge on r815u15n11 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7948(8170649) in scavenge on r815u23n08 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7949(8170650) in scavenge on r816u15n12 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7950(8170651) in scavenge on r816u23n11 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7951(8170652) in scavenge on r918u09n04 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7952(8170653) in scavenge on r909u01n02 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7953(8170654) in scavenge on r909u10n04 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_3(8170655) in scavenge on r807u37n01 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_688(8114957) in scavenge on r802u06n01 Nov 05 10:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_410(7571375) in scavenge on r816u23n09 Nov 05 10:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_7(8168864) WEXITSTATUS 0 Nov 05 10:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_7(8168864) done Nov 05 10:42:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8170656 NodeList=(null) usec=751811 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:42:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:42:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1428(8135961) WEXITSTATUS 0 Nov 05 10:42:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1428(8135961) done Nov 05 10:42:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7851(8169620) WEXITSTATUS 1 Nov 05 10:42:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7851(8169620) done Nov 05 10:43:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_20(8160342) WEXITSTATUS 0 Nov 05 10:43:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_20(8160342) done Nov 05 10:43:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3983(8170642) WEXITSTATUS 0 Nov 05 10:43:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3983(8170642) done Nov 05 10:43:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_411(8169590) WEXITSTATUS 0 Nov 05 10:43:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_411(8169590) done Nov 05 10:43:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_390(8169543) WEXITSTATUS 0 Nov 05 10:43:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_390(8169543) done Nov 05 10:43:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_3(8170655) WEXITSTATUS 1 Nov 05 10:43:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_3(8170655) done Nov 05 10:43:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168560 WEXITSTATUS 0 Nov 05 10:43:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168560 done Nov 05 10:43:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1402(8135935) WEXITSTATUS 0 Nov 05 10:43:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1402(8135935) done Nov 05 10:43:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1996(8164172) WEXITSTATUS 0 Nov 05 10:43:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1996(8164172) done Nov 05 10:43:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_406(8169585) WEXITSTATUS 0 Nov 05 10:43:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_406(8169585) done Nov 05 10:43:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_1(8168867) WEXITSTATUS 0 Nov 05 10:43:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_1(8168867) done Nov 05 10:43:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7547(8166744) WEXITSTATUS 1 Nov 05 10:43:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7547(8166744) done Nov 05 10:43:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7641(8167096) WEXITSTATUS 1 Nov 05 10:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7641(8167096) done Nov 05 10:43:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_396(8169575) WEXITSTATUS 0 Nov 05 10:43:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_396(8169575) done Nov 05 10:43:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:39 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3493(8135505) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:43:39 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7043(8165477) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:43:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_405(8169584) WEXITSTATUS 0 Nov 05 10:43:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_405(8169584) done Nov 05 10:43:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7652(8167257) WEXITSTATUS 1 Nov 05 10:43:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7652(8167257) done Nov 05 10:43:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_408(8169587) WEXITSTATUS 0 Nov 05 10:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_408(8169587) done Nov 05 10:43:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3493(8135505) Nov 05 10:43:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7043(8165477) Nov 05 10:43:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170656 NodeList=r801u30n01 #CPUs=1 Partition=pi_lora Nov 05 10:43:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7461(8166315) WEXITSTATUS 1 Nov 05 10:43:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7461(8166315) done Nov 05 10:43:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_389(8169542) WEXITSTATUS 0 Nov 05 10:43:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_389(8169542) done Nov 05 10:43:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_375(8169528) WEXITSTATUS 0 Nov 05 10:43:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_375(8169528) done Nov 05 10:43:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_499(8169758) WEXITSTATUS 0 Nov 05 10:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_499(8169758) done Nov 05 10:43:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:43:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_409(8169588) WEXITSTATUS 0 Nov 05 10:43:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_409(8169588) done Nov 05 10:43:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_379(8169532) WEXITSTATUS 0 Nov 05 10:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_379(8169532) done Nov 05 10:44:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7950(8170651) WEXITSTATUS 1 Nov 05 10:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7950(8170651) done Nov 05 10:44:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_403(8169582) WEXITSTATUS 0 Nov 05 10:44:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_403(8169582) done Nov 05 10:44:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_378(8169531) WEXITSTATUS 0 Nov 05 10:44:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_378(8169531) done Nov 05 10:44:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170657 InitPrio=46697 usec=7486 Nov 05 10:44:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_3(8168869) WEXITSTATUS 0 Nov 05 10:44:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_3(8168869) done Nov 05 10:44:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_493(8169752) WEXITSTATUS 0 Nov 05 10:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_493(8169752) done Nov 05 10:44:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_407(8169586) WEXITSTATUS 0 Nov 05 10:44:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_407(8169586) done Nov 05 10:44:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_376(8169529) WEXITSTATUS 0 Nov 05 10:44:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_376(8169529) done Nov 05 10:44:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_430(8169609) WEXITSTATUS 0 Nov 05 10:44:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_430(8169609) done Nov 05 10:44:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7646(8167114) WEXITSTATUS 1 Nov 05 10:44:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7646(8167114) done Nov 05 10:44:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_495(8169754) WEXITSTATUS 0 Nov 05 10:44:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_495(8169754) done Nov 05 10:44:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170657 NodeList=r816u35n07 #CPUs=3 Partition=gpu Nov 05 10:44:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_412(8169591) WEXITSTATUS 0 Nov 05 10:44:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_412(8169591) done Nov 05 10:44:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170657 WEXITSTATUS 1 Nov 05 10:44:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170657 done Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:44:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:44:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:44:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_424(8169603) WEXITSTATUS 0 Nov 05 10:44:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_424(8169603) done Nov 05 10:45:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6971(8165317) WEXITSTATUS 1 Nov 05 10:45:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6971(8165317) done Nov 05 10:45:01 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:45:01 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:45:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:02 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:45:03 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:45:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_377(8169530) WEXITSTATUS 0 Nov 05 10:45:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_377(8169530) done Nov 05 10:45:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_2(8168868) WEXITSTATUS 0 Nov 05 10:45:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_2(8168868) done Nov 05 10:45:06 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u02n04.grace.ycrc.yale.edu" Nov 05 10:45:06 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u02n04.grace.ycrc.yale.edu" Nov 05 10:45:07 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u02n03.grace.ycrc.yale.edu" Nov 05 10:45:07 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u02n03.grace.ycrc.yale.edu" Nov 05 10:45:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_433(8169612) WEXITSTATUS 0 Nov 05 10:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_433(8169612) done Nov 05 10:45:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8168558 Nov 05 10:45:09 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u04n03.grace.ycrc.yale.edu" Nov 05 10:45:09 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u04n03.grace.ycrc.yale.edu" Nov 05 10:45:09 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u06n02.grace.ycrc.yale.edu" Nov 05 10:45:09 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u06n02.grace.ycrc.yale.edu" Nov 05 10:45:09 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u04n01.grace.ycrc.yale.edu" Nov 05 10:45:09 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u04n01.grace.ycrc.yale.edu" Nov 05 10:45:09 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u06n01.grace.ycrc.yale.edu" Nov 05 10:45:09 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u06n01.grace.ycrc.yale.edu" Nov 05 10:45:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:11 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u02n02.grace.ycrc.yale.edu" Nov 05 10:45:11 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u02n02.grace.ycrc.yale.edu" Nov 05 10:45:11 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u04n04.grace.ycrc.yale.edu" Nov 05 10:45:11 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u04n04.grace.ycrc.yale.edu" Nov 05 10:45:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_8(8168874) WEXITSTATUS 0 Nov 05 10:45:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_8(8168874) done Nov 05 10:45:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_332(8169425) WEXITSTATUS 0 Nov 05 10:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_332(8169425) done Nov 05 10:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_394(8169573) WEXITSTATUS 0 Nov 05 10:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_394(8169573) done Nov 05 10:45:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170529 WEXITSTATUS 25 Nov 05 10:45:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170529 done Nov 05 10:45:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3984(8170643) WEXITSTATUS 0 Nov 05 10:45:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3984(8170643) done Nov 05 10:45:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_368(8170658) in pi_ohern on r801u32n02 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_369(8170659) in pi_ohern on r917u13n03 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_370(8170660) in pi_ohern on r917u13n04 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_371(8170661) in pi_ohern on r917u13n04 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_372(8170662) in pi_ohern on r917u13n04 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_373(8170663) in pi_ohern on r917u13n04 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_374(8170664) in pi_ohern on r917u29n01 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_375(8170665) in pi_ohern on r917u29n02 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_376(8170666) in pi_ohern on r815u03n11 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_377(8170667) in pi_ohern on r815u03n11 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_378(8170668) in pi_ohern on r815u03n11 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_379(8170669) in pi_ohern on r815u03n11 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_380(8170670) in pi_ohern on r815u03n11 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_381(8170671) in pi_ohern on r815u03n11 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_382(8170672) in pi_ohern on r815u03n11 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_383(8170673) in pi_ohern on r815u03n12 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_384(8170674) in pi_ohern on r815u03n12 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_385(8170675) in pi_ohern on r815u03n12 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_386(8170676) in pi_ohern on r815u03n12 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_387(8170677) in pi_ohern on r911u32n01 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_388(8170678) in pi_ohern on r911u32n01 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_389(8170679) in pi_ohern on r911u32n01 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_390(8170680) in pi_ohern on r913u32n01 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_391(8170681) in pi_ohern on r913u32n01 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_392(8170682) in pi_ohern on r903u24n01 Nov 05 10:45:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_393(8170683) in pi_ohern on r903u26n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_100(8170684) in day on r904u07n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_101(8170685) in day on r904u07n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_102(8170686) in day on r904u07n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_103(8170687) in day on r904u07n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_104(8170688) in day on r904u07n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_105(8170689) in day on r904u10n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_106(8170690) in day on r904u10n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_107(8170691) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_108(8170692) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_109(8170693) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_110(8170694) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_111(8170695) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_112(8170696) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_113(8170697) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_114(8170698) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_115(8170699) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_116(8170700) in day on r914u01n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_117(8170701) in day on r914u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_118(8170702) in day on r914u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_119(8170703) in day on r914u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_120(8170704) in day on r807u08n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_121(8170705) in day on r807u08n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_122(8170706) in day on r807u08n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_123(8170707) in day on r807u08n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_124(8170708) in day on r807u08n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_125(8170709) in day on r807u08n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_126(8170710) in day on r807u08n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_127(8170711) in day on r807u08n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_128(8170712) in day on r807u08n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_129(8170713) in day on r807u08n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_130(8170714) in day on r807u11n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_131(8170715) in day on r807u11n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_132(8170716) in day on r807u11n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_133(8170717) in day on r807u11n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_134(8170718) in day on r807u11n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_135(8170719) in day on r807u11n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_136(8170720) in day on r807u11n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_137(8170721) in day on r807u11n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_138(8170722) in day on r807u11n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_139(8170723) in day on r807u11n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_140(8170724) in day on r807u11n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_141(8170725) in day on r807u11n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_142(8170726) in day on r807u11n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_143(8170727) in day on r807u11n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_144(8170728) in day on r807u11n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_145(8170729) in day on r807u11n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_146(8170730) in day on r807u11n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_147(8170731) in day on r912u28n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_148(8170732) in day on r912u28n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_149(8170733) in day on r912u32n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_150(8170734) in day on r912u32n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_151(8170735) in day on r912u36n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_152(8170736) in day on r912u36n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_153(8170737) in day on r912u36n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_154(8170738) in day on r912u36n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_155(8170739) in day on r912u36n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_156(8170740) in day on r912u36n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_157(8170741) in day on r912u36n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_158(8170742) in day on r912u36n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_159(8170743) in day on r912u36n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_160(8170744) in day on r912u36n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_161(8170745) in day on r912u36n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_162(8170746) in day on r912u36n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_163(8170747) in day on r913u01n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_164(8170748) in day on r913u01n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_165(8170749) in day on r913u01n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_166(8170750) in day on r913u01n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_167(8170751) in day on r913u01n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_168(8170752) in day on r913u05n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_169(8170753) in day on r913u05n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_170(8170754) in day on r913u05n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_171(8170755) in day on r913u05n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_172(8170756) in day on r913u05n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_173(8170757) in day on r913u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_174(8170758) in day on r913u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_175(8170759) in day on r913u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_176(8170760) in day on r913u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_177(8170761) in day on r913u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_178(8170762) in day on r913u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_179(8170763) in day on r913u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_180(8170764) in day on r913u05n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_181(8170765) in day on r913u05n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_182(8170766) in day on r913u05n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_183(8170767) in day on r913u05n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_184(8170768) in day on r913u05n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_185(8170769) in day on r913u09n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_186(8170770) in day on r913u09n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_187(8170771) in day on r913u09n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_188(8170772) in day on r913u09n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_189(8170773) in day on r913u09n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_190(8170774) in day on r913u09n04 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_191(8170775) in day on r913u13n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_192(8170776) in day on r913u13n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_193(8170777) in day on r913u13n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_194(8170778) in day on r913u13n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_195(8170779) in day on r913u13n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_196(8170780) in day on r913u13n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_197(8170781) in day on r913u13n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_198(8170782) in day on r913u13n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_199(8170783) in day on r913u13n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3985(8170784) in gpu on r808u14n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3986(8170785) in gpu on r903u34n01 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r913u13n02 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2934(8170786) in day on r805u06n03 Nov 05 10:45:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2935(8170787) in day on r919u09n03 Nov 05 10:45:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_387(8169540) WEXITSTATUS 0 Nov 05 10:45:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_387(8169540) done Nov 05 10:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_427(8169606) WEXITSTATUS 0 Nov 05 10:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_427(8169606) done Nov 05 10:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2509(8152667) WEXITSTATUS 0 Nov 05 10:45:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2509(8152667) done Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7954(8170788) in scavenge on r802u11n01 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7955(8170789) in scavenge on r904u01n01 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7956(8170790) in scavenge on r815u15n09 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7957(8170791) in scavenge on r816u23n11 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7958(8170792) in scavenge on r816u35n10 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7959(8170793) in scavenge on r4513u26n01 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7960(8170794) in scavenge on r907u28n01 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7961(8170795) in scavenge on r909u13n01 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7962(8170796) in scavenge on r909u13n03 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7963(8170797) in scavenge on r909u13n03 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_4(8170798) in scavenge on r807u37n01 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1011(7571977) in scavenge on r805u06n03 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1079(7572045) in scavenge on r806u02n03 Nov 05 10:45:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1038(7573207) in scavenge on r806u41n01 Nov 05 10:45:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_431(8169610) WEXITSTATUS 0 Nov 05 10:45:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_431(8169610) done Nov 05 10:45:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2453(8165743) in scavenge on r816u29n08 Nov 05 10:45:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_498(8169757) WEXITSTATUS 0 Nov 05 10:45:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_498(8169757) done Nov 05 10:45:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_426(8169605) WEXITSTATUS 0 Nov 05 10:45:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_426(8169605) done Nov 05 10:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_4(8170798) WEXITSTATUS 1 Nov 05 10:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_4(8170798) done Nov 05 10:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_579(8169871) WEXITSTATUS 0 Nov 05 10:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_579(8169871) done Nov 05 10:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_393(8169572) WEXITSTATUS 0 Nov 05 10:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_393(8169572) done Nov 05 10:45:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_8(8158948) WEXITSTATUS 0 Nov 05 10:45:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_8(8158948) done Nov 05 10:45:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3985(8170784) WEXITSTATUS 0 Nov 05 10:45:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3985(8170784) done Nov 05 10:45:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170799 InitPrio=21315 usec=3392 Nov 05 10:45:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_578(8169870) WEXITSTATUS 0 Nov 05 10:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_578(8169870) done Nov 05 10:45:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:45:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_422(8169601) WEXITSTATUS 0 Nov 05 10:45:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_422(8169601) done Nov 05 10:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7387(8166139) WEXITSTATUS 1 Nov 05 10:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7387(8166139) done Nov 05 10:46:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_415(8169594) WEXITSTATUS 0 Nov 05 10:46:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_415(8169594) done Nov 05 10:46:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_423(8169602) WEXITSTATUS 0 Nov 05 10:46:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_423(8169602) done Nov 05 10:46:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7955(8170789) WEXITSTATUS 1 Nov 05 10:46:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7955(8170789) done Nov 05 10:46:08 r912u23n01 slurmctld[163760]: slurmctld: node r803u04n02 returned to service Nov 05 10:46:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_380(8169533) WEXITSTATUS 0 Nov 05 10:46:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_380(8169533) done Nov 05 10:46:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:13 r912u23n01 slurmctld[163760]: slurmctld: node r803u02n01 returned to service Nov 05 10:46:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_494(8169753) WEXITSTATUS 0 Nov 05 10:46:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_494(8169753) done Nov 05 10:46:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7925(8170449) WEXITSTATUS 1 Nov 05 10:46:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7925(8170449) done Nov 05 10:46:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_400(8169579) WEXITSTATUS 0 Nov 05 10:46:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_400(8169579) done Nov 05 10:46:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7947(8170648) WEXITSTATUS 1 Nov 05 10:46:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7947(8170648) done Nov 05 10:46:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_489(8169748) WEXITSTATUS 0 Nov 05 10:46:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_489(8169748) done Nov 05 10:46:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_432(8169611) WEXITSTATUS 0 Nov 05 10:46:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_432(8169611) done Nov 05 10:46:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7919(8170362) WEXITSTATUS 1 Nov 05 10:46:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7919(8170362) done Nov 05 10:46:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_421(8169600) WEXITSTATUS 0 Nov 05 10:46:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_421(8169600) done Nov 05 10:46:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_416(8169595) WEXITSTATUS 0 Nov 05 10:46:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_416(8169595) done Nov 05 10:46:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_497(8169756) WEXITSTATUS 0 Nov 05 10:46:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_497(8169756) done Nov 05 10:46:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_417(8169596) WEXITSTATUS 0 Nov 05 10:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_417(8169596) done Nov 05 10:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_374(8169527) WEXITSTATUS 0 Nov 05 10:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_374(8169527) done Nov 05 10:46:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:46:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_413(8169592) WEXITSTATUS 0 Nov 05 10:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_413(8169592) done Nov 05 10:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_491(8169750) WEXITSTATUS 0 Nov 05 10:46:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_491(8169750) done Nov 05 10:46:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:51 r912u23n01 slurmctld[163760]: slurmctld: node r803u02n03 returned to service Nov 05 10:46:51 r912u23n01 slurmctld[163760]: slurmctld: node r803u02n04 returned to service Nov 05 10:46:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7928(8170452) WEXITSTATUS 1 Nov 05 10:46:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7928(8170452) done Nov 05 10:46:54 r912u23n01 slurmctld[163760]: slurmctld: node r803u04n03 returned to service Nov 05 10:46:54 r912u23n01 slurmctld[163760]: slurmctld: node r803u06n01 returned to service Nov 05 10:46:54 r912u23n01 slurmctld[163760]: slurmctld: node r803u04n01 returned to service Nov 05 10:46:54 r912u23n01 slurmctld[163760]: slurmctld: node r803u06n02 returned to service Nov 05 10:46:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:46:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_388(8169541) WEXITSTATUS 0 Nov 05 10:46:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_388(8169541) done Nov 05 10:46:56 r912u23n01 slurmctld[163760]: slurmctld: node r803u02n02 returned to service Nov 05 10:46:56 r912u23n01 slurmctld[163760]: slurmctld: node r803u04n04 returned to service Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7613(8167058) WEXITSTATUS 1 Nov 05 10:46:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7613(8167058) done Nov 05 10:46:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_455(8169714) WEXITSTATUS 0 Nov 05 10:47:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_455(8169714) done Nov 05 10:47:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907382 WEXITSTATUS 0 Nov 05 10:47:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907382 done Nov 05 10:47:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_429(8169608) WEXITSTATUS 0 Nov 05 10:47:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_429(8169608) done Nov 05 10:47:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_575(8169867) WEXITSTATUS 0 Nov 05 10:47:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_575(8169867) done Nov 05 10:47:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_419(8169598) WEXITSTATUS 0 Nov 05 10:47:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_419(8169598) done Nov 05 10:47:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_505(8169764) WEXITSTATUS 0 Nov 05 10:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_505(8169764) done Nov 05 10:47:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_420(8169599) WEXITSTATUS 0 Nov 05 10:47:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_420(8169599) done Nov 05 10:47:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_490(8169749) WEXITSTATUS 0 Nov 05 10:47:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_490(8169749) done Nov 05 10:47:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_440(8169699) WEXITSTATUS 0 Nov 05 10:47:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_440(8169699) done Nov 05 10:47:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_456(8169715) WEXITSTATUS 0 Nov 05 10:47:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_456(8169715) done Nov 05 10:47:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_479(8169738) WEXITSTATUS 0 Nov 05 10:47:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_479(8169738) done Nov 05 10:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_395(8169574) WEXITSTATUS 0 Nov 05 10:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_395(8169574) done Nov 05 10:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_397(8169576) WEXITSTATUS 0 Nov 05 10:47:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_397(8169576) done Nov 05 10:47:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_516(8169775) WEXITSTATUS 0 Nov 05 10:47:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_516(8169775) done Nov 05 10:47:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_512(8169771) WEXITSTATUS 0 Nov 05 10:47:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_512(8169771) done Nov 05 10:47:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_398(8169577) WEXITSTATUS 0 Nov 05 10:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_398(8169577) done Nov 05 10:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_492(8169751) WEXITSTATUS 0 Nov 05 10:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_492(8169751) done Nov 05 10:47:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_5(8168871) WEXITSTATUS 0 Nov 05 10:47:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_5(8168871) done Nov 05 10:47:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7820(8169457) WEXITSTATUS 1 Nov 05 10:47:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7820(8169457) done Nov 05 10:47:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_428(8169607) WEXITSTATUS 0 Nov 05 10:47:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_428(8169607) done Nov 05 10:47:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7676(8168312) WEXITSTATUS 1 Nov 05 10:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7676(8168312) done Nov 05 10:47:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7734(8168725) WEXITSTATUS 1 Nov 05 10:47:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7734(8168725) done Nov 05 10:47:56 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 10:47:57 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 10:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_580(8169872) WEXITSTATUS 0 Nov 05 10:47:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_580(8169872) done Nov 05 10:47:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_496(8169755) WEXITSTATUS 0 Nov 05 10:47:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_496(8169755) done Nov 05 10:48:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_443(8169702) WEXITSTATUS 0 Nov 05 10:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_443(8169702) done Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_394(8170800) in pi_ohern on r917u13n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_395(8170801) in pi_ohern on r917u13n02 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_396(8170802) in pi_ohern on r917u13n02 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_397(8170803) in pi_ohern on r917u13n03 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_398(8170804) in pi_ohern on r917u13n04 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_399(8170805) in pi_ohern on r917u13n04 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_400(8170806) in pi_ohern on r917u13n04 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_401(8170807) in pi_ohern on r917u13n04 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_402(8170808) in pi_ohern on r917u29n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_403(8170809) in pi_ohern on r917u29n04 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_404(8170810) in pi_ohern on r917u29n04 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_405(8170811) in pi_ohern on r917u29n04 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_406(8170812) in pi_ohern on r815u03n11 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_407(8170813) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_408(8170814) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_409(8170815) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_410(8170816) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_411(8170817) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_412(8170818) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_413(8170819) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_414(8170820) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_415(8170821) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_416(8170822) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_417(8170823) in pi_ohern on r815u03n12 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_418(8170824) in pi_ohern on r911u32n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_419(8170825) in pi_ohern on r911u32n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_420(8170826) in pi_ohern on r911u32n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_421(8170827) in pi_ohern on r911u32n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_422(8170828) in pi_ohern on r911u32n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_423(8170829) in pi_ohern on r807u22n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_424(8170830) in pi_ohern on r807u22n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_425(8170831) in pi_ohern on r807u24n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_426(8170832) in pi_ohern on r913u32n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_427(8170833) in pi_ohern on r913u35n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_428(8170834) in pi_ohern on r913u35n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_429(8170835) in pi_ohern on r903u12n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_430(8170836) in pi_ohern on r903u12n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_431(8170837) in pi_ohern on r903u12n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_432(8170838) in pi_ohern on r903u24n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_433(8170839) in pi_ohern on r903u24n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_434(8170840) in pi_ohern on r903u24n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_435(8170841) in pi_ohern on r903u24n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_436(8170842) in pi_ohern on r903u24n01 Nov 05 10:48:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_437(8170843) in pi_ohern on r903u26n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_200(8170844) in day on r914u05n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_201(8170845) in day on r914u05n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_202(8170846) in day on r914u05n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_203(8170847) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_204(8170848) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_205(8170849) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_206(8170850) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_207(8170851) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_208(8170852) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_209(8170853) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_210(8170854) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_211(8170855) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_212(8170856) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_213(8170857) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_214(8170858) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_215(8170859) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_216(8170860) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_217(8170861) in day on r912u36n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_218(8170862) in day on r913u13n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_219(8170863) in day on r913u13n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_220(8170864) in day on r913u13n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_221(8170865) in day on r913u13n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_222(8170866) in day on r913u24n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_223(8170867) in day on r913u24n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_224(8170868) in day on r913u24n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_225(8170869) in day on r913u24n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_226(8170870) in day on r913u24n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_227(8170871) in day on r913u24n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_228(8170872) in day on r913u24n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_229(8170873) in day on r913u24n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_230(8170874) in day on r913u28n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_231(8170875) in day on r805u02n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_232(8170876) in day on r805u02n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_233(8170877) in day on r805u02n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_234(8170878) in day on r805u02n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_235(8170879) in day on r805u02n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_236(8170880) in day on r805u02n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_237(8170881) in day on r805u02n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_238(8170882) in day on r805u02n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_239(8170883) in day on r805u02n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_240(8170884) in day on r805u02n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_241(8170885) in day on r805u02n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_242(8170886) in day on r805u04n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_243(8170887) in day on r805u04n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_244(8170888) in day on r805u04n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_245(8170889) in day on r805u04n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_246(8170890) in day on r805u04n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_247(8170891) in day on r805u04n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_248(8170892) in day on r805u04n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_249(8170893) in day on r805u04n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_250(8170894) in day on r805u04n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_251(8170895) in day on r805u04n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_252(8170896) in day on r805u04n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_253(8170897) in day on r805u06n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_254(8170898) in day on r805u06n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_255(8170899) in day on r805u06n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_256(8170900) in day on r805u06n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_257(8170901) in day on r805u06n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_258(8170902) in day on r805u06n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_259(8170903) in day on r805u09n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_260(8170904) in day on r805u09n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_261(8170905) in day on r805u09n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_262(8170906) in day on r805u09n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_263(8170907) in day on r805u09n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_264(8170908) in day on r805u11n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_265(8170909) in day on r805u11n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_266(8170910) in day on r805u11n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_267(8170911) in day on r805u11n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_268(8170912) in day on r805u14n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_269(8170913) in day on r805u14n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_270(8170914) in day on r805u23n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_271(8170915) in day on r805u23n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_272(8170916) in day on r805u23n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_273(8170917) in day on r805u23n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_274(8170918) in day on r805u23n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_275(8170919) in day on r805u25n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_276(8170920) in day on r805u25n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_277(8170921) in day on r805u25n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_278(8170922) in day on r805u25n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_279(8170923) in day on r805u25n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_280(8170924) in day on r805u25n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_281(8170925) in day on r805u25n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_282(8170926) in day on r805u25n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_283(8170927) in day on r805u25n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_284(8170928) in day on r805u27n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_285(8170929) in day on r805u27n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_286(8170930) in day on r805u27n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_287(8170931) in day on r805u27n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_288(8170932) in day on r805u27n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_289(8170933) in day on r805u27n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_290(8170934) in day on r909u01n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_291(8170935) in day on r909u01n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_292(8170936) in day on r909u01n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_293(8170937) in day on r909u01n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_294(8170938) in day on r909u01n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_295(8170939) in day on r909u01n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_296(8170940) in day on r909u01n04 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_297(8170941) in day on r909u04n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_298(8170942) in day on r909u04n02 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_299(8170943) in day on r909u04n03 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170799 in gpu on r805u38n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3987(8170944) in gpu on r808u14n01 Nov 05 10:48:09 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:204 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 10:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_3(8168860) WEXITSTATUS 0 Nov 05 10:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168729_3(8168860) done Nov 05 10:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_569(8169861) WEXITSTATUS 0 Nov 05 10:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_569(8169861) done Nov 05 10:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_414(8169593) WEXITSTATUS 0 Nov 05 10:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_414(8169593) done Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7964(8170945) in scavenge on r907u01n03 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7965(8170946) in scavenge on r904u01n01 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7966(8170947) in scavenge on r815u15n09 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7967(8170948) in scavenge on r815u15n11 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7968(8170949) in scavenge on r816u09n10 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7969(8170950) in scavenge on r816u15n09 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7970(8170951) in scavenge on r806u06n04 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7971(8170952) in scavenge on r806u09n01 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7972(8170953) in scavenge on r806u09n04 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7973(8170954) in scavenge on r806u11n01 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_5(8170955) in scavenge on r807u37n01 Nov 05 10:48:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1161(7573330) in scavenge on r805u38n01 Nov 05 10:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_9(8169215) WEXITSTATUS 0 Nov 05 10:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_9(8169215) done Nov 05 10:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6564(8164590) WEXITSTATUS 1 Nov 05 10:48:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6564(8164590) done Nov 05 10:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3987(8170944) WEXITSTATUS 0 Nov 05 10:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3987(8170944) done Nov 05 10:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_504(8169763) WEXITSTATUS 0 Nov 05 10:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_504(8169763) done Nov 05 10:48:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_5(8170955) WEXITSTATUS 1 Nov 05 10:48:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_5(8170955) done Nov 05 10:48:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160530 WEXITSTATUS 0 Nov 05 10:48:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160530 done Nov 05 10:48:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_450(8169709) WEXITSTATUS 0 Nov 05 10:48:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_450(8169709) done Nov 05 10:48:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_515(8169774) WEXITSTATUS 0 Nov 05 10:48:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_515(8169774) done Nov 05 10:48:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_438(8169697) WEXITSTATUS 0 Nov 05 10:48:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_438(8169697) done Nov 05 10:48:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_572(8169864) WEXITSTATUS 0 Nov 05 10:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_572(8169864) done Nov 05 10:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_425(8169604) WEXITSTATUS 0 Nov 05 10:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_425(8169604) done Nov 05 10:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7423(8166212) WEXITSTATUS 1 Nov 05 10:48:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7423(8166212) done Nov 05 10:48:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_10(8169126) WEXITSTATUS 0 Nov 05 10:48:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_10(8169126) done Nov 05 10:48:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_437(8169696) WEXITSTATUS 0 Nov 05 10:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_437(8169696) done Nov 05 10:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_513(8169772) WEXITSTATUS 0 Nov 05 10:48:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_513(8169772) done Nov 05 10:48:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:48:58 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1904(8163114) WEXITSTATUS 0 Nov 05 10:49:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1904(8163114) done Nov 05 10:49:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_441(8169700) WEXITSTATUS 0 Nov 05 10:49:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_441(8169700) done Nov 05 10:49:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7011(8165416) WEXITSTATUS 1 Nov 05 10:49:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7011(8165416) done Nov 05 10:49:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170956 InitPrio=20039 usec=3750 Nov 05 10:49:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169691 WEXITSTATUS 0 Nov 05 10:49:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169691 done Nov 05 10:49:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_511(8169770) WEXITSTATUS 0 Nov 05 10:49:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_511(8169770) done Nov 05 10:49:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_598(8169890) WEXITSTATUS 0 Nov 05 10:49:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_598(8169890) done Nov 05 10:49:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_502(8169761) WEXITSTATUS 0 Nov 05 10:49:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_502(8169761) done Nov 05 10:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_436(8169695) WEXITSTATUS 0 Nov 05 10:49:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_436(8169695) done Nov 05 10:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_465(8169724) WEXITSTATUS 0 Nov 05 10:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_465(8169724) done Nov 05 10:49:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_466(8169725) WEXITSTATUS 0 Nov 05 10:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_466(8169725) done Nov 05 10:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_506(8169765) WEXITSTATUS 0 Nov 05 10:49:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_506(8169765) done Nov 05 10:49:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_574(8169866) WEXITSTATUS 0 Nov 05 10:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_574(8169866) done Nov 05 10:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_435(8169694) WEXITSTATUS 0 Nov 05 10:49:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_435(8169694) done Nov 05 10:49:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_12(8159078) WEXITSTATUS 0 Nov 05 10:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_12(8159078) done Nov 05 10:49:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_392(8169571) WEXITSTATUS 0 Nov 05 10:49:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_392(8169571) done Nov 05 10:49:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_573(8169865) WEXITSTATUS 0 Nov 05 10:49:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_573(8169865) done Nov 05 10:49:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_447(8169706) WEXITSTATUS 0 Nov 05 10:49:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_447(8169706) done Nov 05 10:49:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_439(8169698) WEXITSTATUS 0 Nov 05 10:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_439(8169698) done Nov 05 10:49:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8154208 uid 15915 Nov 05 10:49:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8148355 uid 15915 Nov 05 10:49:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_508(8169767) WEXITSTATUS 0 Nov 05 10:49:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_508(8169767) done Nov 05 10:49:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_503(8169762) WEXITSTATUS 0 Nov 05 10:49:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_503(8169762) done Nov 05 10:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7721(8168547) WEXITSTATUS 1 Nov 05 10:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7721(8168547) done Nov 05 10:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_476(8169735) WEXITSTATUS 0 Nov 05 10:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_476(8169735) done Nov 05 10:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_602(8169894) WEXITSTATUS 0 Nov 05 10:49:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_602(8169894) done Nov 05 10:49:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7938(8170521) WEXITSTATUS 1 Nov 05 10:49:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7938(8170521) done Nov 05 10:49:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_582(8169874) WEXITSTATUS 0 Nov 05 10:49:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_582(8169874) done Nov 05 10:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_457(8169716) WEXITSTATUS 0 Nov 05 10:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_457(8169716) done Nov 05 10:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_481(8169740) WEXITSTATUS 0 Nov 05 10:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_481(8169740) done Nov 05 10:49:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170957 InitPrio=25500 usec=7379 Nov 05 10:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_517(8169776) WEXITSTATUS 0 Nov 05 10:49:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_517(8169776) done Nov 05 10:49:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_571(8169863) WEXITSTATUS 0 Nov 05 10:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_571(8169863) done Nov 05 10:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_399(8169578) WEXITSTATUS 0 Nov 05 10:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_399(8169578) done Nov 05 10:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_586(8169878) WEXITSTATUS 0 Nov 05 10:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_586(8169878) done Nov 05 10:49:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_581(8169873) WEXITSTATUS 0 Nov 05 10:49:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_581(8169873) done Nov 05 10:49:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_471(8169730) WEXITSTATUS 0 Nov 05 10:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_471(8169730) done Nov 05 10:49:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_446(8169705) WEXITSTATUS 0 Nov 05 10:50:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_446(8169705) done Nov 05 10:50:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_459(8169718) WEXITSTATUS 0 Nov 05 10:50:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_459(8169718) done Nov 05 10:50:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_576(8169868) WEXITSTATUS 0 Nov 05 10:50:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_576(8169868) done Nov 05 10:50:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_510(8169769) WEXITSTATUS 0 Nov 05 10:50:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_510(8169769) done Nov 05 10:50:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169121 WEXITSTATUS 0 Nov 05 10:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169121 done Nov 05 10:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1985(8164041) WEXITSTATUS 0 Nov 05 10:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1985(8164041) done Nov 05 10:50:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_583(8169875) WEXITSTATUS 0 Nov 05 10:50:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_583(8169875) done Nov 05 10:50:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_2(8169208) WEXITSTATUS 0 Nov 05 10:50:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_2(8169208) done Nov 05 10:50:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8170958 InitPrio=24068 usec=4821 Nov 05 10:50:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170958 NodeList=r802u23n01 #CPUs=20 Partition=pi_econ_io Nov 05 10:50:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_507(8169766) WEXITSTATUS 0 Nov 05 10:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_507(8169766) done Nov 05 10:50:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_533(8169825) WEXITSTATUS 0 Nov 05 10:50:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_533(8169825) done Nov 05 10:50:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7716(8168529) WEXITSTATUS 1 Nov 05 10:50:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7716(8168529) done Nov 05 10:50:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_475(8169734) WEXITSTATUS 0 Nov 05 10:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_475(8169734) done Nov 05 10:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_484(8169743) WEXITSTATUS 0 Nov 05 10:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_484(8169743) done Nov 05 10:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_528(8169820) WEXITSTATUS 0 Nov 05 10:50:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_528(8169820) done Nov 05 10:50:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_442(8169701) WEXITSTATUS 0 Nov 05 10:50:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_442(8169701) done Nov 05 10:50:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_514(8169773) WEXITSTATUS 0 Nov 05 10:50:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_514(8169773) done Nov 05 10:50:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_500(8169759) WEXITSTATUS 0 Nov 05 10:50:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_500(8169759) done Nov 05 10:50:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_470(8169729) WEXITSTATUS 0 Nov 05 10:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_470(8169729) done Nov 05 10:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154670 WEXITSTATUS 0 Nov 05 10:50:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154670 done Nov 05 10:50:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_501(8169760) WEXITSTATUS 0 Nov 05 10:50:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_501(8169760) done Nov 05 10:50:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_526(8169818) WEXITSTATUS 0 Nov 05 10:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_526(8169818) done Nov 05 10:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_596(8169888) WEXITSTATUS 0 Nov 05 10:50:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_596(8169888) done Nov 05 10:50:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5336(8158955) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_594(8169886) WEXITSTATUS 0 Nov 05 10:50:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_594(8169886) done Nov 05 10:50:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5336(8158955) Nov 05 10:50:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_460(8169719) WEXITSTATUS 0 Nov 05 10:50:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_460(8169719) done Nov 05 10:50:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_509(8169768) WEXITSTATUS 0 Nov 05 10:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_509(8169768) done Nov 05 10:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_485(8169744) WEXITSTATUS 0 Nov 05 10:50:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_485(8169744) done Nov 05 10:50:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_584(8169876) WEXITSTATUS 0 Nov 05 10:50:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_584(8169876) done Nov 05 10:50:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_445(8169704) WEXITSTATUS 0 Nov 05 10:50:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_445(8169704) done Nov 05 10:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7726(8168717) WEXITSTATUS 1 Nov 05 10:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7726(8168717) done Nov 05 10:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3964(8169445) WEXITSTATUS 0 Nov 05 10:50:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3964(8169445) done Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_438(8170959) in pi_ohern on r801u32n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_439(8170960) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_440(8170961) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_441(8170962) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_442(8170963) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_443(8170964) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_444(8170965) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_445(8170966) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_446(8170967) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_447(8170968) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_448(8170969) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_449(8170970) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_450(8170971) in pi_ohern on r917u13n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_451(8170972) in pi_ohern on r917u13n03 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_452(8170973) in pi_ohern on r917u13n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_453(8170974) in pi_ohern on r917u13n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_454(8170975) in pi_ohern on r917u13n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_455(8170976) in pi_ohern on r917u13n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_456(8170977) in pi_ohern on r917u13n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_457(8170978) in pi_ohern on r917u13n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_458(8170979) in pi_ohern on r917u29n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_459(8170980) in pi_ohern on r917u29n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_460(8170981) in pi_ohern on r917u29n02 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_461(8170982) in pi_ohern on r917u29n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_462(8170983) in pi_ohern on r917u29n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_463(8170984) in pi_ohern on r917u29n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_464(8170985) in pi_ohern on r917u29n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_465(8170986) in pi_ohern on r917u29n04 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_466(8170987) in pi_ohern on r815u03n11 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_467(8170988) in pi_ohern on r815u03n12 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_468(8170989) in pi_ohern on r815u03n12 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_469(8170990) in pi_ohern on r815u03n12 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_470(8170991) in pi_ohern on r815u03n12 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_471(8170992) in pi_ohern on r815u03n12 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_472(8170993) in pi_ohern on r815u03n12 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_473(8170994) in pi_ohern on r807u22n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_474(8170995) in pi_ohern on r807u22n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_475(8170996) in pi_ohern on r911u32n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_476(8170997) in pi_ohern on r911u32n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_477(8170998) in pi_ohern on r911u32n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_478(8170999) in pi_ohern on r913u32n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_479(8171000) in pi_ohern on r913u35n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_480(8171001) in pi_ohern on r913u35n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_481(8171002) in pi_ohern on r913u35n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_482(8171003) in pi_ohern on r913u35n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_483(8171004) in pi_ohern on r913u35n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_484(8171005) in pi_ohern on r913u35n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_485(8171006) in pi_ohern on r903u12n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_486(8171007) in pi_ohern on r903u12n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_487(8171008) in pi_ohern on r903u12n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_488(8171009) in pi_ohern on r903u12n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_489(8171010) in pi_ohern on r903u12n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_490(8171011) in pi_ohern on r903u24n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_491(8171012) in pi_ohern on r903u24n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_492(8171013) in pi_ohern on r903u24n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_493(8171014) in pi_ohern on r903u26n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_494(8171015) in pi_ohern on r903u26n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_495(8171016) in pi_ohern on r903u26n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_496(8171017) in pi_ohern on r903u26n01 Nov 05 10:50:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_497(8171018) in pi_ohern on r903u26n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169921 in day on r909u01n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169690 in gpu on r816u35n07,r907u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_300(8171019) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_301(8171020) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_302(8171021) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_303(8171022) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_304(8171023) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_305(8171024) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_306(8171025) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_307(8171026) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_308(8171027) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_309(8171028) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_310(8171029) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_311(8171030) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_312(8171031) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_313(8171032) in day on r914u01n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_314(8171033) in day on r912u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_315(8171034) in day on r912u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_316(8171035) in day on r912u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_317(8171036) in day on r912u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_318(8171037) in day on r912u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_319(8171038) in day on r912u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_320(8171039) in day on r912u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_321(8171040) in day on r912u32n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_322(8171041) in day on r913u28n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_323(8171042) in day on r913u28n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_324(8171043) in day on r909u01n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_325(8171044) in day on r909u04n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_326(8171045) in day on r909u04n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_327(8171046) in day on r909u04n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_328(8171047) in day on r909u07n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_329(8171048) in day on r909u07n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_330(8171049) in day on r909u07n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_331(8171050) in day on r909u07n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_332(8171051) in day on r909u07n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_333(8171052) in day on r909u07n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_334(8171053) in day on r909u07n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_335(8171054) in day on r909u07n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_336(8171055) in day on r909u10n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_337(8171056) in day on r909u10n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_338(8171057) in day on r909u10n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_339(8171058) in day on r909u10n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_340(8171059) in day on r909u10n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_341(8171060) in day on r909u10n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_342(8171061) in day on r909u10n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_343(8171062) in day on r909u10n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_344(8171063) in day on r909u10n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_345(8171064) in day on r909u13n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_346(8171065) in day on r909u13n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_347(8171066) in day on r909u13n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_348(8171067) in day on r909u13n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_349(8171068) in day on r909u13n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_350(8171069) in day on r909u13n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_351(8171070) in day on r909u13n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_352(8171071) in day on r909u13n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_353(8171072) in day on r909u13n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_354(8171073) in day on r909u13n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_355(8171074) in day on r909u13n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_356(8171075) in day on r909u13n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_357(8171076) in day on r909u29n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_358(8171077) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_359(8171078) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_360(8171079) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_361(8171080) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_362(8171081) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_363(8171082) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_364(8171083) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_365(8171084) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_366(8171085) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_367(8171086) in day on r909u29n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_368(8171087) in day on r909u29n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_369(8171088) in day on r909u29n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_370(8171089) in day on r909u29n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_371(8171090) in day on r909u29n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_372(8171091) in day on r806u02n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_373(8171092) in day on r806u02n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_374(8171093) in day on r806u02n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_375(8171094) in day on r806u02n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_376(8171095) in day on r806u02n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_377(8171096) in day on r806u02n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_378(8171097) in day on r806u02n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_379(8171098) in day on r806u02n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_380(8171099) in day on r806u02n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_381(8171100) in day on r806u02n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_382(8171101) in day on r806u02n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_383(8171102) in day on r806u02n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_384(8171103) in day on r806u02n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_385(8171104) in day on r806u02n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_386(8171105) in day on r806u02n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_387(8171106) in day on r806u02n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_388(8171107) in day on r806u02n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_389(8171108) in day on r806u02n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_390(8171109) in day on r806u02n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_391(8171110) in day on r806u02n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_392(8171111) in day on r806u02n04 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_393(8171112) in day on r806u04n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_394(8171113) in day on r806u04n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_395(8171114) in day on r806u04n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_396(8171115) in day on r806u04n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_397(8171116) in day on r806u04n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_398(8171117) in day on r806u04n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_399(8171118) in day on r806u04n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170956 in day on r815u03n[02-03,08],r815u09n[01,04,08],r815u15n08,r815u23n05,r815u29n[02,04-05],r815u35n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3988(8171119) in gpu on r808u14n01 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r806u04n02 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2936(8171120) in day on r816u15n03 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_444(8169703) WEXITSTATUS 0 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_444(8169703) done Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_467(8169726) WEXITSTATUS 0 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_467(8169726) done Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_604(8169896) WEXITSTATUS 0 Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_604(8169896) done Nov 05 10:50:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_449(8169708) WEXITSTATUS 0 Nov 05 10:50:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_449(8169708) done Nov 05 10:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_1(8169980) WEXITSTATUS 0 Nov 05 10:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_1(8169980) done Nov 05 10:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_558(8169850) WEXITSTATUS 0 Nov 05 10:50:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_558(8169850) done Nov 05 10:50:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171121 InitPrio=25500 usec=4025 Nov 05 10:50:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_458(8169717) WEXITSTATUS 0 Nov 05 10:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_458(8169717) done Nov 05 10:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_334(8170486) WEXITSTATUS 0 Nov 05 10:50:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_334(8170486) done Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7974(8171122) in scavenge on r816u15n11 Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7975(8171123) in scavenge on r816u29n10 Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7976(8171124) in scavenge on r815u09n12 Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7977(8171125) in scavenge on r815u15n12 Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7978(8171126) in scavenge on r919u32n04 Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7979(8171127) in scavenge on r919u36n03 Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7980(8171128) in scavenge on r806u11n03 Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7981(8171129) in scavenge on r806u11n04 Nov 05 10:50:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_6(8171130) in scavenge on r807u37n01 Nov 05 10:51:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_8(8169214) WEXITSTATUS 0 Nov 05 10:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_8(8169214) done Nov 05 10:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_464(8169723) WEXITSTATUS 0 Nov 05 10:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_464(8169723) done Nov 05 10:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_477(8169736) WEXITSTATUS 0 Nov 05 10:51:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_477(8169736) done Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:51:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171131 InitPrio=44927 usec=4057 Nov 05 10:51:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_4(8169210) WEXITSTATUS 0 Nov 05 10:51:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_4(8169210) done Nov 05 10:51:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_520(8169812) WEXITSTATUS 0 Nov 05 10:51:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_520(8169812) done Nov 05 10:51:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_7(8169213) WEXITSTATUS 0 Nov 05 10:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_7(8169213) done Nov 05 10:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_6(8169212) WEXITSTATUS 0 Nov 05 10:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_6(8169212) done Nov 05 10:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_6(8171130) WEXITSTATUS 1 Nov 05 10:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_6(8171130) done Nov 05 10:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_577(8169869) WEXITSTATUS 0 Nov 05 10:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_577(8169869) done Nov 05 10:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_591(8169883) WEXITSTATUS 0 Nov 05 10:51:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_591(8169883) done Nov 05 10:51:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_570(8169862) WEXITSTATUS 0 Nov 05 10:51:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_570(8169862) done Nov 05 10:51:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7936(8170519) WEXITSTATUS 1 Nov 05 10:51:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7936(8170519) done Nov 05 10:51:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_530(8169822) WEXITSTATUS 0 Nov 05 10:51:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_530(8169822) done Nov 05 10:51:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_665(8169977) WEXITSTATUS 0 Nov 05 10:51:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_665(8169977) done Nov 05 10:51:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_5(8169211) WEXITSTATUS 0 Nov 05 10:51:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_5(8169211) done Nov 05 10:51:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3986(8170785) WEXITSTATUS 0 Nov 05 10:51:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3986(8170785) done Nov 05 10:51:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_563(8169855) WEXITSTATUS 0 Nov 05 10:51:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_563(8169855) done Nov 05 10:51:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:25 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25292: QOSMaxSubmitJobPerUserLimit Nov 05 10:51:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_454(8169713) WEXITSTATUS 0 Nov 05 10:51:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_454(8169713) done Nov 05 10:51:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:31 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25292: QOSMaxSubmitJobPerUserLimit Nov 05 10:51:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_592(8169884) WEXITSTATUS 0 Nov 05 10:51:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_592(8169884) done Nov 05 10:51:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7344(8166048) WEXITSTATUS 1 Nov 05 10:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7344(8166048) done Nov 05 10:51:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_483(8169742) WEXITSTATUS 0 Nov 05 10:51:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_483(8169742) done Nov 05 10:51:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_478(8169737) WEXITSTATUS 0 Nov 05 10:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_478(8169737) done Nov 05 10:51:43 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25292: QOSMaxSubmitJobPerUserLimit Nov 05 10:51:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 10:51:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7930(8170454) WEXITSTATUS 1 Nov 05 10:51:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7930(8170454) done Nov 05 10:51:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_539(8169831) WEXITSTATUS 0 Nov 05 10:51:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_539(8169831) done Nov 05 10:51:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_607(8169899) WEXITSTATUS 0 Nov 05 10:51:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_607(8169899) done Nov 05 10:51:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7665(8168301) WEXITSTATUS 1 Nov 05 10:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7665(8168301) done Nov 05 10:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_5(8169220) WEXITSTATUS 0 Nov 05 10:51:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_5(8169220) done Nov 05 10:51:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_601(8169893) WEXITSTATUS 0 Nov 05 10:51:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_601(8169893) done Nov 05 10:51:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:51:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7701(8168496) WEXITSTATUS 1 Nov 05 10:51:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7701(8168496) done Nov 05 10:51:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_461(8169720) WEXITSTATUS 0 Nov 05 10:51:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_461(8169720) done Nov 05 10:52:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_529(8169821) WEXITSTATUS 0 Nov 05 10:52:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_529(8169821) done Nov 05 10:52:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_605(8169897) WEXITSTATUS 0 Nov 05 10:52:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_605(8169897) done Nov 05 10:52:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_487(8169746) WEXITSTATUS 0 Nov 05 10:52:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_487(8169746) done Nov 05 10:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_463(8169722) WEXITSTATUS 0 Nov 05 10:52:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_463(8169722) done Nov 05 10:52:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171132 InitPrio=53844 usec=3902 Nov 05 10:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_448(8169707) WEXITSTATUS 0 Nov 05 10:52:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_448(8169707) done Nov 05 10:52:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_418(8169597) WEXITSTATUS 0 Nov 05 10:52:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_418(8169597) done Nov 05 10:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_8(8169223) WEXITSTATUS 0 Nov 05 10:52:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_8(8169223) done Nov 05 10:52:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_666(8169978) WEXITSTATUS 0 Nov 05 10:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_666(8169978) done Nov 05 10:52:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:52:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_661(8169973) WEXITSTATUS 0 Nov 05 10:52:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_661(8169973) done Nov 05 10:52:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_462(8169721) WEXITSTATUS 0 Nov 05 10:52:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_462(8169721) done Nov 05 10:52:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7082(8165616) WEXITSTATUS 1 Nov 05 10:52:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7082(8165616) done Nov 05 10:52:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_597(8169889) WEXITSTATUS 0 Nov 05 10:52:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_597(8169889) done Nov 05 10:52:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_531(8169823) WEXITSTATUS 0 Nov 05 10:52:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_531(8169823) done Nov 05 10:52:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_451(8169710) WEXITSTATUS 0 Nov 05 10:52:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_451(8169710) done Nov 05 10:52:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_603(8169895) WEXITSTATUS 0 Nov 05 10:52:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_603(8169895) done Nov 05 10:52:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_588(8169880) WEXITSTATUS 0 Nov 05 10:52:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_588(8169880) done Nov 05 10:52:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_473(8169732) WEXITSTATUS 0 Nov 05 10:52:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_473(8169732) done Nov 05 10:52:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_474(8169733) WEXITSTATUS 0 Nov 05 10:52:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_474(8169733) done Nov 05 10:52:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_609(8169901) WEXITSTATUS 0 Nov 05 10:52:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_609(8169901) done Nov 05 10:52:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_472(8169731) WEXITSTATUS 0 Nov 05 10:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_472(8169731) done Nov 05 10:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_468(8169727) WEXITSTATUS 0 Nov 05 10:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_468(8169727) done Nov 05 10:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_545(8169837) WEXITSTATUS 0 Nov 05 10:52:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_545(8169837) done Nov 05 10:52:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6019(8163029) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:52:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7460(8166314) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:52:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2684(8166485) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:52:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6019(8163029) Nov 05 10:52:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7460(8166314) Nov 05 10:52:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2684(8166485) Nov 05 10:52:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_452(8169711) WEXITSTATUS 0 Nov 05 10:52:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_452(8169711) done Nov 05 10:52:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_71(8170065) WEXITSTATUS 0 Nov 05 10:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_71(8170065) done Nov 05 10:52:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:52:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171131 NodeList=r903u34n01 #CPUs=3 Partition=gpu Nov 05 10:52:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_519(8169811) WEXITSTATUS 0 Nov 05 10:52:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_519(8169811) done Nov 05 10:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_587(8169879) WEXITSTATUS 0 Nov 05 10:52:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_587(8169879) done Nov 05 10:52:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_524(8169816) WEXITSTATUS 0 Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_524(8169816) done Nov 05 10:53:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171133 InitPrio=31021 usec=5743 Nov 05 10:53:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_595(8169887) WEXITSTATUS 0 Nov 05 10:53:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_595(8169887) done Nov 05 10:53:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_606(8169898) WEXITSTATUS 0 Nov 05 10:53:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_606(8169898) done Nov 05 10:53:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_547(8169839) WEXITSTATUS 0 Nov 05 10:53:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_547(8169839) done Nov 05 10:53:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_518(8169810) WEXITSTATUS 0 Nov 05 10:53:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_518(8169810) done Nov 05 10:53:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_527(8169819) WEXITSTATUS 0 Nov 05 10:53:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_527(8169819) done Nov 05 10:53:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_534(8169826) WEXITSTATUS 0 Nov 05 10:53:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_534(8169826) done Nov 05 10:53:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171131 WEXITSTATUS 1 Nov 05 10:53:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171131 done Nov 05 10:53:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_10(8168791) WEXITSTATUS 0 Nov 05 10:53:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_10(8168791) done Nov 05 10:53:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_593(8169885) WEXITSTATUS 0 Nov 05 10:53:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_593(8169885) done Nov 05 10:53:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_664(8169976) WEXITSTATUS 0 Nov 05 10:53:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_664(8169976) done Nov 05 10:53:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_542(8169834) WEXITSTATUS 0 Nov 05 10:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_542(8169834) done Nov 05 10:53:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_627(8169939) WEXITSTATUS 0 Nov 05 10:53:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_627(8169939) done Nov 05 10:53:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_70(8170064) WEXITSTATUS 0 Nov 05 10:53:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_70(8170064) done Nov 05 10:53:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_4(8169983) WEXITSTATUS 0 Nov 05 10:53:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_4(8169983) done Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_498(8171134) in pi_ohern on r801u32n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_499(8171135) in pi_ohern on r801u32n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_500(8171136) in pi_ohern on r801u32n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_501(8171137) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_502(8171138) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_503(8171139) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_504(8171140) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_505(8171141) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_506(8171142) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_507(8171143) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_508(8171144) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_509(8171145) in pi_ohern on r917u13n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_510(8171146) in pi_ohern on r917u13n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_511(8171147) in pi_ohern on r917u13n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_512(8171148) in pi_ohern on r917u13n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_513(8171149) in pi_ohern on r917u13n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_514(8171150) in pi_ohern on r917u13n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_515(8171151) in pi_ohern on r917u13n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_516(8171152) in pi_ohern on r917u13n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_517(8171153) in pi_ohern on r917u13n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_518(8171154) in pi_ohern on r917u13n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_519(8171155) in pi_ohern on r917u29n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_520(8171156) in pi_ohern on r917u29n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_521(8171157) in pi_ohern on r917u29n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_522(8171158) in pi_ohern on r917u29n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_523(8171159) in pi_ohern on r917u29n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_524(8171160) in pi_ohern on r917u29n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_525(8171161) in pi_ohern on r917u29n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_526(8171162) in pi_ohern on r917u29n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_527(8171163) in pi_ohern on r917u29n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_528(8171164) in pi_ohern on r917u29n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_529(8171165) in pi_ohern on r917u29n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_530(8171166) in pi_ohern on r917u29n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_531(8171167) in pi_ohern on r917u29n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_532(8171168) in pi_ohern on r917u29n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_533(8171169) in pi_ohern on r917u29n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_534(8171170) in pi_ohern on r917u29n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_535(8171171) in pi_ohern on r815u03n11 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_536(8171172) in pi_ohern on r815u03n11 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_537(8171173) in pi_ohern on r815u03n12 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_538(8171174) in pi_ohern on r815u03n12 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_539(8171175) in pi_ohern on r911u32n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_540(8171176) in pi_ohern on r911u32n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_541(8171177) in pi_ohern on r913u35n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_542(8171178) in pi_ohern on r807u22n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_543(8171179) in pi_ohern on r807u22n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_544(8171180) in pi_ohern on r807u22n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_545(8171181) in pi_ohern on r807u24n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_546(8171182) in pi_ohern on r807u24n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_547(8171183) in pi_ohern on r903u12n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_548(8171184) in pi_ohern on r903u12n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_549(8171185) in pi_ohern on r903u12n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_550(8171186) in pi_ohern on r903u12n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_551(8171187) in pi_ohern on r903u12n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_552(8171188) in pi_ohern on r903u24n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_553(8171189) in pi_ohern on r903u26n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_554(8171190) in pi_ohern on r903u26n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_555(8171191) in pi_ohern on r903u26n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_556(8171192) in pi_ohern on r903u26n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_557(8171193) in pi_ohern on r903u26n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_558(8171194) in pi_ohern on r903u26n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_559(8171195) in pi_ohern on r903u26n01 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_400(8171196) in day on r807u05n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_401(8171197) in day on r807u05n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_402(8171198) in day on r806u04n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_403(8171199) in day on r806u04n02 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_404(8171200) in day on r806u04n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_405(8171201) in day on r806u04n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_406(8171202) in day on r806u04n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_407(8171203) in day on r806u04n03 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_408(8171204) in day on r806u04n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_409(8171205) in day on r806u04n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_410(8171206) in day on r806u04n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_411(8171207) in day on r806u04n04 Nov 05 10:53:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_412(8171208) in day on r806u04n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_413(8171209) in day on r806u06n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_414(8171210) in day on r806u06n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_415(8171211) in day on r806u06n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_416(8171212) in day on r806u06n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_417(8171213) in day on r806u06n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_418(8171214) in day on r806u06n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_419(8171215) in day on r806u06n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_420(8171216) in day on r806u06n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_421(8171217) in day on r806u06n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_422(8171218) in day on r806u06n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_423(8171219) in day on r806u06n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_424(8171220) in day on r806u06n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_425(8171221) in day on r806u06n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_426(8171222) in day on r806u06n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_427(8171223) in day on r806u06n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_428(8171224) in day on r806u06n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_429(8171225) in day on r806u09n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_430(8171226) in day on r806u09n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_431(8171227) in day on r806u09n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_432(8171228) in day on r806u09n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_433(8171229) in day on r806u09n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_434(8171230) in day on r806u09n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_435(8171231) in day on r806u09n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_436(8171232) in day on r806u09n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_437(8171233) in day on r806u09n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_438(8171234) in day on r806u09n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_439(8171235) in day on r806u09n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_440(8171236) in day on r806u09n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_441(8171237) in day on r806u09n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_442(8171238) in day on r806u09n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_443(8171239) in day on r806u09n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_444(8171240) in day on r806u09n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_445(8171241) in day on r806u09n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_446(8171242) in day on r806u11n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_447(8171243) in day on r806u11n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_448(8171244) in day on r806u11n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_449(8171245) in day on r806u11n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_450(8171246) in day on r806u11n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_451(8171247) in day on r806u11n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_452(8171248) in day on r806u11n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_453(8171249) in day on r806u11n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_454(8171250) in day on r806u11n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_455(8171251) in day on r806u11n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_456(8171252) in day on r806u11n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_457(8171253) in day on r806u11n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_458(8171254) in day on r806u11n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_459(8171255) in day on r806u11n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_460(8171256) in day on r806u14n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_461(8171257) in day on r806u14n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_462(8171258) in day on r806u14n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_463(8171259) in day on r806u14n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_464(8171260) in day on r806u14n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_465(8171261) in day on r806u14n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_466(8171262) in day on r806u14n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_467(8171263) in day on r806u14n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_468(8171264) in day on r806u14n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_469(8171265) in day on r806u14n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_470(8171266) in day on r806u14n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_471(8171267) in day on r806u14n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_472(8171268) in day on r806u14n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_473(8171269) in day on r806u14n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_474(8171270) in day on r806u14n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_475(8171271) in day on r806u14n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_476(8171272) in day on r806u14n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_477(8171273) in day on r806u14n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_478(8171274) in day on r806u14n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_479(8171275) in day on r806u14n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_480(8171276) in day on r806u23n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_481(8171277) in day on r806u23n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_482(8171278) in day on r918u01n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_483(8171279) in day on r918u01n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_484(8171280) in day on r918u01n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_485(8171281) in day on r918u01n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_486(8171282) in day on r918u01n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_487(8171283) in day on r918u01n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_488(8171284) in day on r918u01n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_489(8171285) in day on r918u01n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_490(8171286) in day on r918u01n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_491(8171287) in day on r918u01n02 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_492(8171288) in day on r918u01n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_493(8171289) in day on r918u01n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_494(8171290) in day on r918u01n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_495(8171291) in day on r918u01n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_496(8171292) in day on r918u01n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_497(8171293) in day on r918u01n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_498(8171294) in day on r918u01n03 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_499(8171295) in day on r918u01n04 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3989(8171296) in gpu on r903u34n01 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:176 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_17(8169996) WEXITSTATUS 0 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_17(8169996) done Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_585(8169877) WEXITSTATUS 0 Nov 05 10:53:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_585(8169877) done Nov 05 10:53:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2861(8128507) Nov 05 10:53:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3059(8129766) Nov 05 10:53:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_610(8169922) WEXITSTATUS 0 Nov 05 10:53:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_610(8169922) done Nov 05 10:53:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7982(8171297) in scavenge on r802u11n02 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7983(8171298) in scavenge on r904u01n02 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7984(8171299) in scavenge on r903u34n01 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7985(8171300) in scavenge on r903u34n01 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7986(8171301) in scavenge on r919u05n04 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7987(8171302) in scavenge on r919u09n01 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7988(8171303) in scavenge on r815u35n02 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7989(8171304) in scavenge on r816u23n06 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7990(8171305) in scavenge on r816u09n08 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7991(8171306) in scavenge on r816u15n02 Nov 05 10:53:42 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_7(8171307) in scavenge on r807u37n01 Nov 05 10:53:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:43 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 10:53:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_550(8169842) WEXITSTATUS 0 Nov 05 10:53:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_550(8169842) done Nov 05 10:53:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2937(8171308) in scavenge on r903u34n01 Nov 05 10:53:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_653(8169965) WEXITSTATUS 0 Nov 05 10:53:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_653(8169965) done Nov 05 10:53:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_7(8171307) WEXITSTATUS 1 Nov 05 10:53:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_7(8171307) done Nov 05 10:53:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_13(8169992) WEXITSTATUS 0 Nov 05 10:53:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_13(8169992) done Nov 05 10:53:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_538(8169830) WEXITSTATUS 0 Nov 05 10:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_538(8169830) done Nov 05 10:53:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_599(8169891) WEXITSTATUS 0 Nov 05 10:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_599(8169891) done Nov 05 10:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_480(8169739) WEXITSTATUS 0 Nov 05 10:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_480(8169739) done Nov 05 10:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_523(8169815) WEXITSTATUS 0 Nov 05 10:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_523(8169815) done Nov 05 10:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_9(8169988) WEXITSTATUS 0 Nov 05 10:53:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_9(8169988) done Nov 05 10:53:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_667(8169979) WEXITSTATUS 0 Nov 05 10:54:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_667(8169979) done Nov 05 10:54:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_434(8169693) WEXITSTATUS 0 Nov 05 10:54:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_434(8169693) done Nov 05 10:54:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_469(8169728) WEXITSTATUS 0 Nov 05 10:54:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_469(8169728) done Nov 05 10:54:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_482(8169741) WEXITSTATUS 0 Nov 05 10:54:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_482(8169741) done Nov 05 10:54:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3989(8171296) WEXITSTATUS 0 Nov 05 10:54:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3989(8171296) done Nov 05 10:54:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_521(8169813) WEXITSTATUS 0 Nov 05 10:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_521(8169813) done Nov 05 10:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_488(8169747) WEXITSTATUS 0 Nov 05 10:54:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_488(8169747) done Nov 05 10:54:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_600(8169892) WEXITSTATUS 0 Nov 05 10:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_600(8169892) done Nov 05 10:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_590(8169882) WEXITSTATUS 0 Nov 05 10:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_590(8169882) done Nov 05 10:54:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_557(8169849) WEXITSTATUS 0 Nov 05 10:54:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_557(8169849) done Nov 05 10:54:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_536(8169828) WEXITSTATUS 0 Nov 05 10:54:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_536(8169828) done Nov 05 10:54:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_69(8170063) WEXITSTATUS 0 Nov 05 10:54:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_69(8170063) done Nov 05 10:54:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_14(8169993) WEXITSTATUS 0 Nov 05 10:54:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_14(8169993) done Nov 05 10:54:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_565(8169857) WEXITSTATUS 0 Nov 05 10:54:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_565(8169857) done Nov 05 10:54:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_611(8169923) WEXITSTATUS 0 Nov 05 10:54:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_611(8169923) done Nov 05 10:54:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_81(8170075) WEXITSTATUS 0 Nov 05 10:54:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_81(8170075) done Nov 05 10:54:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_554(8169846) WEXITSTATUS 0 Nov 05 10:54:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_554(8169846) done Nov 05 10:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_535(8169827) WEXITSTATUS 0 Nov 05 10:54:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_535(8169827) done Nov 05 10:54:21 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 10:54:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_3(8169982) WEXITSTATUS 0 Nov 05 10:54:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_3(8169982) done Nov 05 10:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_6(8169985) WEXITSTATUS 0 Nov 05 10:54:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_6(8169985) done Nov 05 10:54:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_589(8169881) WEXITSTATUS 0 Nov 05 10:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_589(8169881) done Nov 05 10:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_83(8170077) WEXITSTATUS 0 Nov 05 10:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_83(8170077) done Nov 05 10:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_630(8169942) WEXITSTATUS 0 Nov 05 10:54:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_630(8169942) done Nov 05 10:54:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_663(8169975) WEXITSTATUS 0 Nov 05 10:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_663(8169975) done Nov 05 10:54:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_11(8169990) WEXITSTATUS 0 Nov 05 10:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_11(8169990) done Nov 05 10:54:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171309 InitPrio=34224 usec=3976 Nov 05 10:54:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_7(8169986) WEXITSTATUS 0 Nov 05 10:54:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_7(8169986) done Nov 05 10:54:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_68(8170062) WEXITSTATUS 0 Nov 05 10:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_68(8170062) done Nov 05 10:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_532(8169824) WEXITSTATUS 0 Nov 05 10:54:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_532(8169824) done Nov 05 10:54:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_546(8169838) WEXITSTATUS 0 Nov 05 10:54:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_546(8169838) done Nov 05 10:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_76(8170070) WEXITSTATUS 0 Nov 05 10:54:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_76(8170070) done Nov 05 10:54:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_634(8169946) WEXITSTATUS 0 Nov 05 10:54:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_634(8169946) done Nov 05 10:54:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_82(8170076) WEXITSTATUS 0 Nov 05 10:54:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_82(8170076) done Nov 05 10:54:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6520(8164458) WEXITSTATUS 1 Nov 05 10:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6520(8164458) done Nov 05 10:54:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_564(8169856) WEXITSTATUS 0 Nov 05 10:54:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_564(8169856) done Nov 05 10:54:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_656(8169968) WEXITSTATUS 0 Nov 05 10:54:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_656(8169968) done Nov 05 10:54:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_553(8169845) WEXITSTATUS 0 Nov 05 10:54:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_553(8169845) done Nov 05 10:54:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_618(8169930) WEXITSTATUS 0 Nov 05 10:54:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_618(8169930) done Nov 05 10:54:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_544(8169836) WEXITSTATUS 0 Nov 05 10:54:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_544(8169836) done Nov 05 10:54:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144135 WEXITSTATUS 0 Nov 05 10:54:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144135 done Nov 05 10:54:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_562(8169854) WEXITSTATUS 0 Nov 05 10:54:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_562(8169854) done Nov 05 10:54:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:54:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_453(8169712) WEXITSTATUS 0 Nov 05 10:54:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_453(8169712) done Nov 05 10:55:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_560(8169852) WEXITSTATUS 0 Nov 05 10:55:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_560(8169852) done Nov 05 10:55:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_567(8169859) WEXITSTATUS 0 Nov 05 10:55:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_567(8169859) done Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:55:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:55:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_540(8169832) WEXITSTATUS 0 Nov 05 10:55:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_540(8169832) done Nov 05 10:55:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_646(8169958) WEXITSTATUS 0 Nov 05 10:55:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_646(8169958) done Nov 05 10:55:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_655(8169967) WEXITSTATUS 0 Nov 05 10:55:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_655(8169967) done Nov 05 10:55:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_78(8170072) WEXITSTATUS 0 Nov 05 10:55:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_78(8170072) done Nov 05 10:55:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_10(8169989) WEXITSTATUS 0 Nov 05 10:55:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_10(8169989) done Nov 05 10:55:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_654(8169966) WEXITSTATUS 0 Nov 05 10:55:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_654(8169966) done Nov 05 10:55:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_551(8169843) WEXITSTATUS 0 Nov 05 10:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_551(8169843) done Nov 05 10:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_72(8170066) WEXITSTATUS 0 Nov 05 10:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_72(8170066) done Nov 05 10:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7707(8168510) WEXITSTATUS 1 Nov 05 10:55:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7707(8168510) done Nov 05 10:55:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_626(8169938) WEXITSTATUS 0 Nov 05 10:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_626(8169938) done Nov 05 10:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_628(8169940) WEXITSTATUS 0 Nov 05 10:55:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_628(8169940) done Nov 05 10:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_620(8169932) WEXITSTATUS 0 Nov 05 10:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_620(8169932) done Nov 05 10:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_537(8169829) WEXITSTATUS 0 Nov 05 10:55:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_537(8169829) done Nov 05 10:55:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_543(8169835) WEXITSTATUS 0 Nov 05 10:55:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_543(8169835) done Nov 05 10:55:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160529 WEXITSTATUS 0 Nov 05 10:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160529 done Nov 05 10:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3988(8171119) WEXITSTATUS 0 Nov 05 10:55:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3988(8171119) done Nov 05 10:55:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8171309 uid 26297 Nov 05 10:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_552(8169844) WEXITSTATUS 0 Nov 05 10:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_552(8169844) done Nov 05 10:55:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6544(8164497) WEXITSTATUS 1 Nov 05 10:55:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6544(8164497) done Nov 05 10:55:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7565(8166780) WEXITSTATUS 1 Nov 05 10:55:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7565(8166780) done Nov 05 10:55:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_74(8170068) WEXITSTATUS 0 Nov 05 10:55:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_74(8170068) done Nov 05 10:55:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_549(8169841) WEXITSTATUS 0 Nov 05 10:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_549(8169841) done Nov 05 10:55:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_647(8169959) WEXITSTATUS 0 Nov 05 10:55:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_647(8169959) done Nov 05 10:55:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_652(8169964) WEXITSTATUS 0 Nov 05 10:55:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_652(8169964) done Nov 05 10:55:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_85(8170079) WEXITSTATUS 0 Nov 05 10:55:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_85(8170079) done Nov 05 10:55:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_559(8169851) WEXITSTATUS 0 Nov 05 10:55:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_559(8169851) done Nov 05 10:55:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_100(8170094) WEXITSTATUS 0 Nov 05 10:55:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_100(8170094) done Nov 05 10:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_615(8169927) WEXITSTATUS 0 Nov 05 10:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_615(8169927) done Nov 05 10:55:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7704(8168499) WEXITSTATUS 1 Nov 05 10:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7704(8168499) done Nov 05 10:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_8(8169987) WEXITSTATUS 0 Nov 05 10:55:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_8(8169987) done Nov 05 10:55:40 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:41 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:41 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:42 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:43 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:43 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:44 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:46 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 10:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_73(8170067) WEXITSTATUS 0 Nov 05 10:55:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_73(8170067) done Nov 05 10:55:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_2(8169217) WEXITSTATUS 0 Nov 05 10:55:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_2(8169217) done Nov 05 10:55:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_621(8169933) WEXITSTATUS 0 Nov 05 10:55:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_621(8169933) done Nov 05 10:55:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_12(8169991) WEXITSTATUS 0 Nov 05 10:55:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_12(8169991) done Nov 05 10:55:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7794(8169271) WEXITSTATUS 1 Nov 05 10:55:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7794(8169271) done Nov 05 10:55:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_2(8169981) WEXITSTATUS 0 Nov 05 10:55:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_2(8169981) done Nov 05 10:55:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144161 NodeList=r801u04n02,r801u09n01 #CPUs=72 Partition=scavenge Nov 05 10:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_66(8170060) WEXITSTATUS 0 Nov 05 10:55:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_66(8170060) done Nov 05 10:55:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_98(8170092) WEXITSTATUS 0 Nov 05 10:55:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_98(8170092) done Nov 05 10:55:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_525(8169817) WEXITSTATUS 0 Nov 05 10:55:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_525(8169817) done Nov 05 10:55:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_5(8169984) WEXITSTATUS 0 Nov 05 10:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_5(8169984) done Nov 05 10:55:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_608(8169900) WEXITSTATUS 0 Nov 05 10:55:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_608(8169900) done Nov 05 10:56:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_658(8169970) WEXITSTATUS 0 Nov 05 10:56:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_658(8169970) done Nov 05 10:56:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:56:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1994(8164170) WEXITSTATUS 0 Nov 05 10:56:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1994(8164170) done Nov 05 10:56:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_16(8169995) WEXITSTATUS 0 Nov 05 10:56:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_16(8169995) done Nov 05 10:56:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_561(8169853) WEXITSTATUS 0 Nov 05 10:56:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_561(8169853) done Nov 05 10:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_80(8170074) WEXITSTATUS 0 Nov 05 10:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_80(8170074) done Nov 05 10:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_625(8169937) WEXITSTATUS 0 Nov 05 10:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_625(8169937) done Nov 05 10:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_522(8169814) WEXITSTATUS 0 Nov 05 10:56:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_522(8169814) done Nov 05 10:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_612(8169924) WEXITSTATUS 0 Nov 05 10:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_612(8169924) done Nov 05 10:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_84(8170078) WEXITSTATUS 0 Nov 05 10:56:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_84(8170078) done Nov 05 10:56:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:56:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_555(8169847) WEXITSTATUS 0 Nov 05 10:56:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_555(8169847) done Nov 05 10:56:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169690 WEXITSTATUS 0 Nov 05 10:56:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169690 done Nov 05 10:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_79(8170073) WEXITSTATUS 0 Nov 05 10:56:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_79(8170073) done Nov 05 10:56:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:56:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:56:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171310 InitPrio=44927 usec=4479 Nov 05 10:56:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_1(8169216) WEXITSTATUS 0 Nov 05 10:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_1(8169216) done Nov 05 10:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_77(8170071) WEXITSTATUS 0 Nov 05 10:56:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_77(8170071) done Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_560(8171311) in pi_ohern on r801u32n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_561(8171312) in pi_ohern on r801u32n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_562(8171313) in pi_ohern on r801u32n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_563(8171314) in pi_ohern on r801u32n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_564(8171315) in pi_ohern on r801u32n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_565(8171316) in pi_ohern on r917u13n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_566(8171317) in pi_ohern on r917u13n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_567(8171318) in pi_ohern on r917u13n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_568(8171319) in pi_ohern on r917u13n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_569(8171320) in pi_ohern on r917u13n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_570(8171321) in pi_ohern on r917u13n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_571(8171322) in pi_ohern on r917u13n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_572(8171323) in pi_ohern on r917u13n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_573(8171324) in pi_ohern on r917u13n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_574(8171325) in pi_ohern on r917u13n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_575(8171326) in pi_ohern on r917u13n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_576(8171327) in pi_ohern on r917u13n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_577(8171328) in pi_ohern on r917u13n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_578(8171329) in pi_ohern on r917u29n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_579(8171330) in pi_ohern on r917u29n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_580(8171331) in pi_ohern on r917u29n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_581(8171332) in pi_ohern on r917u29n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_582(8171333) in pi_ohern on r917u29n02 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_583(8171334) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_584(8171335) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_585(8171336) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_586(8171337) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_587(8171338) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_588(8171339) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_589(8171340) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_590(8171341) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_591(8171342) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_592(8171343) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_593(8171344) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_594(8171345) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_595(8171346) in pi_ohern on r917u29n03 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_596(8171347) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_597(8171348) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_598(8171349) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_599(8171350) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_600(8171351) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_601(8171352) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_602(8171353) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_603(8171354) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_604(8171355) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_605(8171356) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_606(8171357) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_607(8171358) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_608(8171359) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_609(8171360) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_610(8171361) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_611(8171362) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_612(8171363) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_613(8171364) in pi_ohern on r917u29n04 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_614(8171365) in pi_ohern on r815u03n11 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_615(8171366) in pi_ohern on r815u03n11 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_616(8171367) in pi_ohern on r815u03n11 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_617(8171368) in pi_ohern on r815u03n11 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_618(8171369) in pi_ohern on r815u03n11 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_619(8171370) in pi_ohern on r815u03n11 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_620(8171371) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_621(8171372) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_622(8171373) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_623(8171374) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_624(8171375) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_625(8171376) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_626(8171377) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_627(8171378) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_628(8171379) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_629(8171380) in pi_ohern on r815u03n12 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_630(8171381) in pi_ohern on r807u22n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_631(8171382) in pi_ohern on r807u24n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_632(8171383) in pi_ohern on r807u24n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_633(8171384) in pi_ohern on r807u24n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_634(8171385) in pi_ohern on r903u12n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_635(8171386) in pi_ohern on r903u12n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_636(8171387) in pi_ohern on r903u26n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_637(8171388) in pi_ohern on r903u26n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_638(8171389) in pi_ohern on r903u26n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_639(8171390) in pi_ohern on r903u26n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_640(8171391) in pi_ohern on r903u26n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_641(8171392) in pi_ohern on r903u26n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_642(8171393) in pi_ohern on r913u32n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_643(8171394) in pi_ohern on r913u32n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_644(8171395) in pi_ohern on r913u32n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_645(8171396) in pi_ohern on r913u32n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_646(8171397) in pi_ohern on r913u32n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_647(8171398) in pi_ohern on r913u32n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_648(8171399) in pi_ohern on r913u32n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_649(8171400) in pi_ohern on r913u35n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_650(8171401) in pi_ohern on r913u35n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_651(8171402) in pi_ohern on r913u35n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_652(8171403) in pi_ohern on r913u35n01 Nov 05 10:56:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_653(8171404) in pi_ohern on r913u35n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8171132 in day on r914u01n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8171310 in gpu on r816u35n07 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_500(8171405) in day on r805u14n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_501(8171406) in day on r805u14n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_502(8171407) in day on r805u14n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_503(8171408) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_504(8171409) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_505(8171410) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_506(8171411) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_507(8171412) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_508(8171413) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_509(8171414) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_510(8171415) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_511(8171416) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_512(8171417) in day on r914u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_513(8171418) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_514(8171419) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_515(8171420) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_516(8171421) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_517(8171422) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_518(8171423) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_519(8171424) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_520(8171425) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_521(8171426) in day on r918u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_522(8171427) in day on r918u05n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_523(8171428) in day on r918u05n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_524(8171429) in day on r918u05n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_525(8171430) in day on r918u05n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_526(8171431) in day on r918u05n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_527(8171432) in day on r918u05n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_528(8171433) in day on r918u05n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_529(8171434) in day on r918u05n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_530(8171435) in day on r918u09n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_531(8171436) in day on r918u09n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_532(8171437) in day on r918u09n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_533(8171438) in day on r918u09n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_534(8171439) in day on r918u09n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_535(8171440) in day on r918u09n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_536(8171441) in day on r918u09n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_537(8171442) in day on r918u09n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_538(8171443) in day on r918u09n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_539(8171444) in day on r918u09n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_540(8171445) in day on r918u32n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_541(8171446) in day on r918u32n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_542(8171447) in day on r918u32n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_543(8171448) in day on r918u32n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_544(8171449) in day on r918u32n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_545(8171450) in day on r918u32n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_546(8171451) in day on r918u36n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_547(8171452) in day on r918u36n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_548(8171453) in day on r918u36n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_549(8171454) in day on r918u36n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_550(8171455) in day on r918u36n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_551(8171456) in day on r918u36n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_552(8171457) in day on r919u01n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_553(8171458) in day on r919u01n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_554(8171459) in day on r919u01n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_555(8171460) in day on r919u01n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_556(8171461) in day on r919u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_557(8171462) in day on r919u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_558(8171463) in day on r919u01n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_559(8171464) in day on r919u05n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_560(8171465) in day on r919u05n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_561(8171466) in day on r919u05n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_562(8171467) in day on r919u05n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_563(8171468) in day on r919u05n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_564(8171469) in day on r919u09n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_565(8171470) in day on r919u09n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_566(8171471) in day on r919u09n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_567(8171472) in day on r919u09n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_568(8171473) in day on r919u09n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_569(8171474) in day on r919u32n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_570(8171475) in day on r919u32n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_571(8171476) in day on r919u32n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_572(8171477) in day on r919u32n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_573(8171478) in day on r919u36n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_574(8171479) in day on r919u36n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_575(8171480) in day on r919u36n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_576(8171481) in day on r919u36n02 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_577(8171482) in day on r919u36n03 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_578(8171483) in day on r919u36n04 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_579(8171484) in day on r815u23n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_580(8171485) in day on r815u23n01 Nov 05 10:56:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_581(8171486) in day on r815u23n01 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_582(8171487) in day on r815u23n01 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_583(8171488) in day on r815u23n02 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_584(8171489) in day on r815u23n02 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_585(8171490) in day on r815u23n02 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_586(8171491) in day on r815u23n02 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_587(8171492) in day on r815u23n05 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_588(8171493) in day on r815u23n06 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_589(8171494) in day on r815u29n01 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_590(8171495) in day on r815u29n01 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_591(8171496) in day on r815u29n01 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_592(8171497) in day on r815u29n02 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_593(8171498) in day on r815u29n02 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_594(8171499) in day on r815u29n03 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_595(8171500) in day on r815u29n04 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_596(8171501) in day on r815u29n04 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_597(8171502) in day on r815u29n04 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_598(8171503) in day on r815u29n04 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_599(8171504) in day on r815u29n04 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3990(8171505) in gpu on r808u14n01 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3991(8171506) in gpu on r903u34n01 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3992(8171507) in gpu on r907u32n01 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r815u29n04 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7675(8168311) WEXITSTATUS 1 Nov 05 10:56:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7675(8168311) done Nov 05 10:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_486(8169745) WEXITSTATUS 0 Nov 05 10:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_486(8169745) done Nov 05 10:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_541(8169833) WEXITSTATUS 0 Nov 05 10:56:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_541(8169833) done Nov 05 10:56:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7992(8171508) in scavenge on r801u09n04 Nov 05 10:56:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7993(8171509) in scavenge on r802u06n02 Nov 05 10:56:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7994(8171510) in scavenge on r816u03n09 Nov 05 10:56:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7995(8171511) in scavenge on r816u09n10 Nov 05 10:56:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7996(8171512) in scavenge on r907u32n01 Nov 05 10:56:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7997(8171513) in scavenge on r816u15n03 Nov 05 10:56:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7998(8171514) in scavenge on r816u15n03 Nov 05 10:56:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_8(8171515) in scavenge on r807u37n01 Nov 05 10:56:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_638(8169950) WEXITSTATUS 0 Nov 05 10:56:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_638(8169950) done Nov 05 10:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_102(8170096) WEXITSTATUS 0 Nov 05 10:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_102(8170096) done Nov 05 10:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7616(8167061) WEXITSTATUS 1 Nov 05 10:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7616(8167061) done Nov 05 10:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_622(8169934) WEXITSTATUS 0 Nov 05 10:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_622(8169934) done Nov 05 10:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_659(8169971) WEXITSTATUS 0 Nov 05 10:56:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_659(8169971) done Nov 05 10:56:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_8(8171515) WEXITSTATUS 1 Nov 05 10:56:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_8(8171515) done Nov 05 10:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_15(8169994) WEXITSTATUS 0 Nov 05 10:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_15(8169994) done Nov 05 10:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_641(8169953) WEXITSTATUS 0 Nov 05 10:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_641(8169953) done Nov 05 10:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171310 WEXITSTATUS 1 Nov 05 10:56:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171310 done Nov 05 10:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_568(8169860) WEXITSTATUS 0 Nov 05 10:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_568(8169860) done Nov 05 10:56:39 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8171516 NodeList=(null) usec=22216 Nov 05 10:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_662(8169974) WEXITSTATUS 0 Nov 05 10:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_662(8169974) done Nov 05 10:56:40 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171516 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 10:56:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_645(8169957) WEXITSTATUS 0 Nov 05 10:56:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_645(8169957) done Nov 05 10:56:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7542(8166674) WEXITSTATUS 1 Nov 05 10:56:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7542(8166674) done Nov 05 10:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_633(8169945) WEXITSTATUS 0 Nov 05 10:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_633(8169945) done Nov 05 10:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_566(8169858) WEXITSTATUS 0 Nov 05 10:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_566(8169858) done Nov 05 10:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_651(8169963) WEXITSTATUS 0 Nov 05 10:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_651(8169963) done Nov 05 10:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_614(8169926) WEXITSTATUS 0 Nov 05 10:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_614(8169926) done Nov 05 10:56:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171517 InitPrio=34224 usec=3634 Nov 05 10:56:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_101(8170095) WEXITSTATUS 0 Nov 05 10:56:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_101(8170095) done Nov 05 10:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_623(8169935) WEXITSTATUS 0 Nov 05 10:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_623(8169935) done Nov 05 10:56:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171517 NodeList=r816u23n05 #CPUs=4 Partition=week Nov 05 10:56:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169918 WEXITSTATUS 0 Nov 05 10:56:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169918 done Nov 05 10:56:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3979(8170512) WEXITSTATUS 0 Nov 05 10:56:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3979(8170512) done Nov 05 10:56:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_99(8170093) WEXITSTATUS 0 Nov 05 10:56:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_99(8170093) done Nov 05 10:56:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_624(8169936) WEXITSTATUS 0 Nov 05 10:56:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_624(8169936) done Nov 05 10:56:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_643(8169955) WEXITSTATUS 0 Nov 05 10:56:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_643(8169955) done Nov 05 10:56:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:56:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:56:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168540 WEXITSTATUS 0 Nov 05 10:56:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168540 done Nov 05 10:57:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_67(8170061) WEXITSTATUS 0 Nov 05 10:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_67(8170061) done Nov 05 10:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_88(8170082) WEXITSTATUS 0 Nov 05 10:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_88(8170082) done Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:57:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:57:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_650(8169962) WEXITSTATUS 0 Nov 05 10:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_650(8169962) done Nov 05 10:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_75(8170069) WEXITSTATUS 0 Nov 05 10:57:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_75(8170069) done Nov 05 10:57:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_32(8170026) WEXITSTATUS 0 Nov 05 10:57:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_32(8170026) done Nov 05 10:57:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7960(8170794) WEXITSTATUS 1 Nov 05 10:57:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7960(8170794) done Nov 05 10:57:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_556(8169848) WEXITSTATUS 0 Nov 05 10:57:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_556(8169848) done Nov 05 10:57:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3195(8132249) Nov 05 10:57:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_613(8169925) WEXITSTATUS 0 Nov 05 10:57:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_613(8169925) done Nov 05 10:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7865(8169784) WEXITSTATUS 1 Nov 05 10:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7865(8169784) done Nov 05 10:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_164(8170175) WEXITSTATUS 0 Nov 05 10:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_164(8170175) done Nov 05 10:57:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_103(8170097) WEXITSTATUS 0 Nov 05 10:57:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_103(8170097) done Nov 05 10:57:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_632(8169944) WEXITSTATUS 0 Nov 05 10:57:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_632(8169944) done Nov 05 10:57:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_619(8169931) WEXITSTATUS 0 Nov 05 10:57:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_619(8169931) done Nov 05 10:57:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_165(8170176) WEXITSTATUS 0 Nov 05 10:57:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_165(8170176) done Nov 05 10:57:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_657(8169969) WEXITSTATUS 0 Nov 05 10:57:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_657(8169969) done Nov 05 10:57:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_41(8170035) WEXITSTATUS 0 Nov 05 10:57:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_41(8170035) done Nov 05 10:57:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_87(8170081) WEXITSTATUS 0 Nov 05 10:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_87(8170081) done Nov 05 10:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_91(8170085) WEXITSTATUS 0 Nov 05 10:57:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_91(8170085) done Nov 05 10:57:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_94(8170088) WEXITSTATUS 0 Nov 05 10:57:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_94(8170088) done Nov 05 10:57:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 10:57:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_96(8170090) WEXITSTATUS 0 Nov 05 10:57:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_96(8170090) done Nov 05 10:57:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_53(8170047) WEXITSTATUS 0 Nov 05 10:57:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_53(8170047) done Nov 05 10:57:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:57:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_38(8170032) WEXITSTATUS 0 Nov 05 10:57:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_38(8170032) done Nov 05 10:57:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_635(8169947) WEXITSTATUS 0 Nov 05 10:58:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_635(8169947) done Nov 05 10:58:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1403(8135936) WEXITSTATUS 0 Nov 05 10:58:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1403(8135936) done Nov 05 10:58:13 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 10:58:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 10:58:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 10:58:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 10:58:16 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 10:58:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7553(8166750) WEXITSTATUS 1 Nov 05 10:58:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7553(8166750) done Nov 05 10:58:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_54(8170048) WEXITSTATUS 0 Nov 05 10:58:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_54(8170048) done Nov 05 10:58:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_93(8170087) WEXITSTATUS 0 Nov 05 10:58:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_93(8170087) done Nov 05 10:58:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7360(8166093) WEXITSTATUS 1 Nov 05 10:58:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7360(8166093) done Nov 05 10:58:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_56(8170050) WEXITSTATUS 0 Nov 05 10:58:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_56(8170050) done Nov 05 10:58:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_90(8170084) WEXITSTATUS 0 Nov 05 10:58:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_90(8170084) done Nov 05 10:58:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_35(8170029) WEXITSTATUS 0 Nov 05 10:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_35(8170029) done Nov 05 10:58:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_649(8169961) WEXITSTATUS 0 Nov 05 10:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_649(8169961) done Nov 05 10:58:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_9(8168875) WEXITSTATUS 0 Nov 05 10:58:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168791_9(8168875) done Nov 05 10:58:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5458(8159774) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:58:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7937(8170520) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:58:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7939(8170522) has been requeued to reclaim resources for JobId=Invalid Nov 05 10:58:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7939(8170522) Nov 05 10:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_89(8170083) WEXITSTATUS 0 Nov 05 10:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_89(8170083) done Nov 05 10:58:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5458(8159774) Nov 05 10:58:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_61(8170055) WEXITSTATUS 0 Nov 05 10:58:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_61(8170055) done Nov 05 10:58:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7937(8170520) Nov 05 10:58:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7832013_0(7832013) Nov 05 10:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_92(8170086) WEXITSTATUS 0 Nov 05 10:58:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_92(8170086) done Nov 05 10:58:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7966(8170947) WEXITSTATUS 1 Nov 05 10:58:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7966(8170947) done Nov 05 10:58:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_167(8170178) WEXITSTATUS 0 Nov 05 10:58:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_167(8170178) done Nov 05 10:58:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_33(8170027) WEXITSTATUS 0 Nov 05 10:58:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_33(8170027) done Nov 05 10:58:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_637(8169949) WEXITSTATUS 0 Nov 05 10:58:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_637(8169949) done Nov 05 10:58:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171133 NodeList=r919u32n04 #CPUs=15 Partition=day Nov 05 10:58:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170957 NodeList=r805u23n02 #CPUs=24 Partition=day Nov 05 10:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_95(8170089) WEXITSTATUS 0 Nov 05 10:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_95(8170089) done Nov 05 10:58:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_629(8169941) WEXITSTATUS 0 Nov 05 10:58:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_629(8169941) done Nov 05 10:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_1(8169207) WEXITSTATUS 0 Nov 05 10:58:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_1(8169207) done Nov 05 10:58:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 10:58:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_86(8170080) WEXITSTATUS 0 Nov 05 10:58:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_86(8170080) done Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_654(8171518) in pi_ohern on r917u13n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_655(8171519) in pi_ohern on r917u13n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_656(8171520) in pi_ohern on r917u13n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_657(8171521) in pi_ohern on r917u13n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_658(8171522) in pi_ohern on r917u13n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_659(8171523) in pi_ohern on r917u13n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_660(8171524) in pi_ohern on r917u13n02 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_661(8171525) in pi_ohern on r917u13n02 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_662(8171526) in pi_ohern on r917u13n02 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_663(8171527) in pi_ohern on r917u13n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_664(8171528) in pi_ohern on r917u13n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_665(8171529) in pi_ohern on r917u13n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_666(8171530) in pi_ohern on r917u13n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_667(8171531) in pi_ohern on r917u13n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8168792_668(8168792) in pi_ohern on r917u29n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_1(8171532) in pi_ohern on r917u29n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_2(8171533) in pi_ohern on r917u29n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_3(8171534) in pi_ohern on r917u29n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_4(8171535) in pi_ohern on r917u29n02 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_5(8171536) in pi_ohern on r917u29n02 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_6(8171537) in pi_ohern on r917u29n02 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_7(8171538) in pi_ohern on r917u29n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_8(8171539) in pi_ohern on r917u29n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_9(8171540) in pi_ohern on r917u29n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_10(8171541) in pi_ohern on r917u29n03 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_11(8171542) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_12(8171543) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_13(8171544) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_14(8171545) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_15(8171546) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_16(8171547) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_17(8171548) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_18(8171549) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_19(8171550) in pi_ohern on r917u29n04 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_20(8171551) in pi_ohern on r815u03n11 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_21(8171552) in pi_ohern on r815u03n11 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_22(8171553) in pi_ohern on r815u03n11 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_23(8171554) in pi_ohern on r815u03n12 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_24(8171555) in pi_ohern on r807u22n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_25(8171556) in pi_ohern on r807u22n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_26(8171557) in pi_ohern on r807u22n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_27(8171558) in pi_ohern on r807u22n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_28(8171559) in pi_ohern on r807u22n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_29(8171560) in pi_ohern on r903u12n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_30(8171561) in pi_ohern on r903u12n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_31(8171562) in pi_ohern on r903u12n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_32(8171563) in pi_ohern on r911u32n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_33(8171564) in pi_ohern on r911u32n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_34(8171565) in pi_ohern on r911u32n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_35(8171566) in pi_ohern on r911u32n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_36(8171567) in pi_ohern on r911u32n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_37(8171568) in pi_ohern on r913u32n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_38(8171569) in pi_ohern on r913u35n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_39(8171570) in pi_ohern on r913u35n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_40(8171571) in pi_ohern on r913u35n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_41(8171572) in pi_ohern on r913u35n01 Nov 05 10:59:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_600(8171573) in day on r919u32n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_601(8171574) in day on r919u32n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_602(8171575) in day on r805u06n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_603(8171576) in day on r805u06n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_604(8171577) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_605(8171578) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_606(8171579) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_607(8171580) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_608(8171581) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_609(8171582) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_610(8171583) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_611(8171584) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_612(8171585) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_613(8171586) in day on r805u23n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_614(8171587) in day on r805u25n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_615(8171588) in day on r805u25n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_616(8171589) in day on r805u25n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_617(8171590) in day on r815u29n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_618(8171591) in day on r815u29n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_619(8171592) in day on r815u29n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_620(8171593) in day on r815u29n06 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_621(8171594) in day on r815u35n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_622(8171595) in day on r815u35n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_623(8171596) in day on r815u35n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_624(8171597) in day on r815u35n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_625(8171598) in day on r815u35n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_626(8171599) in day on r815u35n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_627(8171600) in day on r815u35n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_628(8171601) in day on r815u35n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_629(8171602) in day on r815u35n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_630(8171603) in day on r815u03n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_631(8171604) in day on r815u03n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_632(8171605) in day on r815u03n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_633(8171606) in day on r815u03n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_634(8171607) in day on r815u03n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_635(8171608) in day on r815u03n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_636(8171609) in day on r815u03n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_637(8171610) in day on r815u03n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_638(8171611) in day on r815u03n07 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_639(8171612) in day on r815u03n07 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_640(8171613) in day on r815u03n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_641(8171614) in day on r815u03n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_642(8171615) in day on r815u03n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_643(8171616) in day on r815u09n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_644(8171617) in day on r815u09n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_645(8171618) in day on r815u09n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_646(8171619) in day on r815u09n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_647(8171620) in day on r815u09n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_648(8171621) in day on r815u09n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_649(8171622) in day on r815u09n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_650(8171623) in day on r815u09n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_651(8171624) in day on r815u09n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_652(8171625) in day on r815u09n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_653(8171626) in day on r815u09n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_654(8171627) in day on r815u09n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_655(8171628) in day on r815u09n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_656(8171629) in day on r815u09n06 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_657(8171630) in day on r815u09n06 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_658(8171631) in day on r815u09n06 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_659(8171632) in day on r815u09n06 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_660(8171633) in day on r815u09n06 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_661(8171634) in day on r815u09n07 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_662(8171635) in day on r815u09n07 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_663(8171636) in day on r815u09n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_664(8171637) in day on r815u09n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_665(8171638) in day on r815u09n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_666(8171639) in day on r815u15n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_667(8171640) in day on r815u15n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_668(8171641) in day on r815u15n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_669(8171642) in day on r815u15n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_670(8171643) in day on r815u15n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_671(8171644) in day on r815u15n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_672(8171645) in day on r815u15n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_673(8171646) in day on r815u15n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_674(8171647) in day on r815u15n06 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_675(8171648) in day on r815u15n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_676(8171649) in day on r815u15n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_677(8171650) in day on r816u03n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_678(8171651) in day on r816u03n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_679(8171652) in day on r816u03n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_680(8171653) in day on r816u03n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_681(8171654) in day on r816u03n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_682(8171655) in day on r816u03n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_683(8171656) in day on r816u03n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_684(8171657) in day on r816u03n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_685(8171658) in day on r816u03n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_686(8171659) in day on r816u03n03 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_687(8171660) in day on r816u03n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_688(8171661) in day on r816u03n04 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_689(8171662) in day on r816u03n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_690(8171663) in day on r816u03n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_691(8171664) in day on r816u03n05 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_692(8171665) in day on r816u03n06 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_693(8171666) in day on r816u03n07 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_694(8171667) in day on r816u03n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_695(8171668) in day on r816u03n08 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_696(8171669) in day on r816u09n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_697(8171670) in day on r816u09n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_698(8171671) in day on r816u09n02 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3993(8171672) in gpu on r808u08n01 Nov 05 10:59:01 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:198 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 10:59:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_42(8170036) WEXITSTATUS 0 Nov 05 10:59:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_42(8170036) done Nov 05 10:59:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_29(8170023) WEXITSTATUS 0 Nov 05 10:59:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_29(8170023) done Nov 05 10:59:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_46(8170040) WEXITSTATUS 0 Nov 05 10:59:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_46(8170040) done Nov 05 10:59:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_175(8170186) WEXITSTATUS 0 Nov 05 10:59:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_175(8170186) done Nov 05 10:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_31(8170025) WEXITSTATUS 0 Nov 05 10:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_31(8170025) done Nov 05 10:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_47(8170041) WEXITSTATUS 0 Nov 05 10:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_47(8170041) done Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_7999(8171673) in scavenge on r801u30n01 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8000(8171674) in scavenge on r907u05n02 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8001(8171675) in scavenge on r903u01n02 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8002(8171676) in scavenge on r815u15n09 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8003(8171677) in scavenge on r816u29n09 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8004(8171678) in scavenge on r816u35n11 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8005(8171679) in scavenge on r907u28n01 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8006(8171680) in scavenge on r907u34n01 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8007(8171681) in scavenge on r816u15n06 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8008(8171682) in scavenge on r816u15n06 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8009(8171683) in scavenge on r816u15n07 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8140415_9(8171684) in scavenge on r807u37n01 Nov 05 10:59:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_971(7573140) in scavenge on r816u29n09 Nov 05 10:59:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:700 Nov 05 10:59:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3198(8132315) Nov 05 10:59:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8169123 Nov 05 10:59:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2938(8171685) in scavenge on r816u29n11 Nov 05 10:59:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2939(8171686) in scavenge on r902u07n01 Nov 05 10:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7730(8168721) WEXITSTATUS 1 Nov 05 10:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7730(8168721) done Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_64(8170058) WEXITSTATUS 0 Nov 05 10:59:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_64(8170058) done Nov 05 10:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_65(8170059) WEXITSTATUS 0 Nov 05 10:59:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_65(8170059) done Nov 05 10:59:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_51(8170045) WEXITSTATUS 0 Nov 05 10:59:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_51(8170045) done Nov 05 10:59:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_9(8171684) WEXITSTATUS 1 Nov 05 10:59:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8140415_9(8171684) done Nov 05 10:59:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8171687 NodeList=(null) usec=29149 Nov 05 10:59:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171687 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 10:59:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_58(8170052) WEXITSTATUS 0 Nov 05 10:59:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_58(8170052) done Nov 05 10:59:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_631(8169943) WEXITSTATUS 0 Nov 05 10:59:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_631(8169943) done Nov 05 10:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159214 WEXITSTATUS 0 Nov 05 10:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159214 done Nov 05 10:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_221(8170327) WEXITSTATUS 0 Nov 05 10:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_221(8170327) done Nov 05 10:59:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_173(8170184) WEXITSTATUS 0 Nov 05 10:59:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_173(8170184) done Nov 05 10:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_26(8170020) WEXITSTATUS 0 Nov 05 10:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_26(8170020) done Nov 05 10:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_648(8169960) WEXITSTATUS 0 Nov 05 10:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_648(8169960) done Nov 05 10:59:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_168(8170179) WEXITSTATUS 0 Nov 05 10:59:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_168(8170179) done Nov 05 10:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_62(8170056) WEXITSTATUS 0 Nov 05 10:59:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_62(8170056) done Nov 05 10:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_616(8169928) WEXITSTATUS 0 Nov 05 10:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_616(8169928) done Nov 05 10:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_97(8170091) WEXITSTATUS 0 Nov 05 10:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_97(8170091) done Nov 05 10:59:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_57(8170051) WEXITSTATUS 0 Nov 05 10:59:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_57(8170051) done Nov 05 10:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_182(8170193) WEXITSTATUS 0 Nov 05 10:59:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_182(8170193) done Nov 05 10:59:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_354(8170506) WEXITSTATUS 0 Nov 05 10:59:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_354(8170506) done Nov 05 10:59:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_617(8169929) WEXITSTATUS 0 Nov 05 10:59:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_617(8169929) done Nov 05 10:59:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7973(8170954) WEXITSTATUS 1 Nov 05 10:59:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7973(8170954) done Nov 05 10:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_36(8170030) WEXITSTATUS 0 Nov 05 10:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_36(8170030) done Nov 05 10:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_639(8169951) WEXITSTATUS 0 Nov 05 10:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_639(8169951) done Nov 05 10:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_134(8170145) WEXITSTATUS 0 Nov 05 10:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_134(8170145) done Nov 05 10:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_166(8170177) WEXITSTATUS 0 Nov 05 10:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_166(8170177) done Nov 05 10:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_27(8170021) WEXITSTATUS 0 Nov 05 10:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_27(8170021) done Nov 05 10:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_169(8170180) WEXITSTATUS 0 Nov 05 10:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_169(8170180) done Nov 05 10:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_28(8170022) WEXITSTATUS 0 Nov 05 10:59:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_28(8170022) done Nov 05 10:59:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_644(8169956) WEXITSTATUS 0 Nov 05 10:59:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_644(8169956) done Nov 05 11:00:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_21(8170015) WEXITSTATUS 0 Nov 05 11:00:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_21(8170015) done Nov 05 11:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 11:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 11:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 11:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 11:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 11:00:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_45(8170039) WEXITSTATUS 0 Nov 05 11:00:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_45(8170039) done Nov 05 11:00:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_347(8170499) WEXITSTATUS 0 Nov 05 11:00:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_347(8170499) done Nov 05 11:00:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3200(8132317) Nov 05 11:00:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_174(8170185) WEXITSTATUS 0 Nov 05 11:00:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_174(8170185) done Nov 05 11:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_48(8170042) WEXITSTATUS 0 Nov 05 11:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_48(8170042) done Nov 05 11:00:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_43(8170037) WEXITSTATUS 0 Nov 05 11:00:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_43(8170037) done Nov 05 11:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_37(8170031) WEXITSTATUS 0 Nov 05 11:00:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_37(8170031) done Nov 05 11:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_63(8170057) WEXITSTATUS 0 Nov 05 11:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_63(8170057) done Nov 05 11:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8006(8171680) WEXITSTATUS 1 Nov 05 11:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8006(8171680) done Nov 05 11:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7861(8169780) WEXITSTATUS 1 Nov 05 11:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7861(8169780) done Nov 05 11:00:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_19(8170013) WEXITSTATUS 0 Nov 05 11:00:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_19(8170013) done Nov 05 11:00:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_60(8170054) WEXITSTATUS 0 Nov 05 11:00:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_60(8170054) done Nov 05 11:00:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_34(8170028) WEXITSTATUS 0 Nov 05 11:00:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_34(8170028) done Nov 05 11:00:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_30(8170024) WEXITSTATUS 0 Nov 05 11:00:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_30(8170024) done Nov 05 11:00:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_25(8170019) WEXITSTATUS 0 Nov 05 11:00:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_25(8170019) done Nov 05 11:00:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_55(8170049) WEXITSTATUS 0 Nov 05 11:00:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_55(8170049) done Nov 05 11:00:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_548(8169840) WEXITSTATUS 0 Nov 05 11:00:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_548(8169840) done Nov 05 11:00:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3950(8169188) WEXITSTATUS 0 Nov 05 11:00:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3950(8169188) done Nov 05 11:00:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_50(8170044) WEXITSTATUS 0 Nov 05 11:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_50(8170044) done Nov 05 11:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_3(8169218) WEXITSTATUS 0 Nov 05 11:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_3(8169218) done Nov 05 11:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7783(8169191) WEXITSTATUS 1 Nov 05 11:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7783(8169191) done Nov 05 11:00:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_286(8170423) WEXITSTATUS 0 Nov 05 11:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_286(8170423) done Nov 05 11:00:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_636(8169948) WEXITSTATUS 0 Nov 05 11:00:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_636(8169948) done Nov 05 11:00:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:00:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r806u11n01 #CPUs=1 Partition=day Nov 05 11:00:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r806u11n01 #CPUs=1 Partition=day Nov 05 11:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7295(8165975) WEXITSTATUS 1 Nov 05 11:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7295(8165975) done Nov 05 11:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_10(8169129) WEXITSTATUS 0 Nov 05 11:00:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_10(8169129) done Nov 05 11:00:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171688 InitPrio=27319 usec=8287 Nov 05 11:01:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_9(8169224) WEXITSTATUS 0 Nov 05 11:01:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_9(8169224) done Nov 05 11:01:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_24(8170018) WEXITSTATUS 0 Nov 05 11:01:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_24(8170018) done Nov 05 11:01:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_59(8170053) WEXITSTATUS 0 Nov 05 11:01:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_59(8170053) done Nov 05 11:01:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_178(8170189) WEXITSTATUS 0 Nov 05 11:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_178(8170189) done Nov 05 11:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_39(8170033) WEXITSTATUS 0 Nov 05 11:01:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_39(8170033) done Nov 05 11:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_349(8170501) WEXITSTATUS 0 Nov 05 11:01:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_349(8170501) done Nov 05 11:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_224(8170330) WEXITSTATUS 0 Nov 05 11:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_224(8170330) done Nov 05 11:01:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171689 InitPrio=17549 usec=7039 Nov 05 11:01:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_180(8170191) WEXITSTATUS 0 Nov 05 11:01:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_180(8170191) done Nov 05 11:01:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171689 NodeList=r801u04n01 #CPUs=12 Partition=pi_esi Nov 05 11:01:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_343(8170495) WEXITSTATUS 0 Nov 05 11:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_343(8170495) done Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:01:14 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:01:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7406(8166176) WEXITSTATUS 1 Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7406(8166176) done Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_171(8170182) WEXITSTATUS 0 Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_171(8170182) done Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7877(8169909) WEXITSTATUS 1 Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7877(8169909) done Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_642(8169954) WEXITSTATUS 0 Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_642(8169954) done Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_172(8170183) WEXITSTATUS 0 Nov 05 11:01:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_172(8170183) done Nov 05 11:01:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_4(8169219) WEXITSTATUS 0 Nov 05 11:01:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_4(8169219) done Nov 05 11:01:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_291(8170428) WEXITSTATUS 0 Nov 05 11:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_291(8170428) done Nov 05 11:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_223(8170329) WEXITSTATUS 0 Nov 05 11:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_223(8170329) done Nov 05 11:01:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_660(8169972) WEXITSTATUS 0 Nov 05 11:01:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_660(8169972) done Nov 05 11:01:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 11:01:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 11:01:27 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 11:01:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171690 InitPrio=27319 usec=3632 Nov 05 11:01:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171691 InitPrio=44927 usec=4165 Nov 05 11:01:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_640(8169952) WEXITSTATUS 0 Nov 05 11:01:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168730_640(8169952) done Nov 05 11:01:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171692 InitPrio=61795 usec=8374 Nov 05 11:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7984(8171299) WEXITSTATUS 1 Nov 05 11:01:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7984(8171299) done Nov 05 11:01:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_52(8170046) WEXITSTATUS 0 Nov 05 11:01:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_52(8170046) done Nov 05 11:01:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3978(8170346) WEXITSTATUS 0 Nov 05 11:01:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3978(8170346) done Nov 05 11:01:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7863(8169782) WEXITSTATUS 1 Nov 05 11:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7863(8169782) done Nov 05 11:01:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8171690 uid 25787 Nov 05 11:01:41 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2939(8171686) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:01:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_6(8169221) WEXITSTATUS 0 Nov 05 11:01:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_6(8169221) done Nov 05 11:01:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2939(8171686) Nov 05 11:01:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_18(8170012) WEXITSTATUS 0 Nov 05 11:01:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_18(8170012) done Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_42(8171693) in pi_ohern on r917u13n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_43(8171694) in pi_ohern on r917u13n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_44(8171695) in pi_ohern on r917u13n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_45(8171696) in pi_ohern on r917u13n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_46(8171697) in pi_ohern on r917u13n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_47(8171698) in pi_ohern on r917u13n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_48(8171699) in pi_ohern on r917u13n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_49(8171700) in pi_ohern on r917u13n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_50(8171701) in pi_ohern on r917u13n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_51(8171702) in pi_ohern on r917u13n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_52(8171703) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_53(8171704) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_54(8171705) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_55(8171706) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_56(8171707) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_57(8171708) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_58(8171709) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_59(8171710) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_60(8171711) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_61(8171712) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_62(8171713) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_63(8171714) in pi_ohern on r917u13n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_64(8171715) in pi_ohern on r917u29n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_65(8171716) in pi_ohern on r917u29n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_66(8171717) in pi_ohern on r917u29n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_67(8171718) in pi_ohern on r917u29n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_68(8171719) in pi_ohern on r917u29n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_69(8171720) in pi_ohern on r917u29n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_70(8171721) in pi_ohern on r917u29n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_71(8171722) in pi_ohern on r917u29n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_72(8171723) in pi_ohern on r917u29n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_73(8171724) in pi_ohern on r917u29n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_74(8171725) in pi_ohern on r917u29n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_75(8171726) in pi_ohern on r917u29n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_76(8171727) in pi_ohern on r917u29n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_77(8171728) in pi_ohern on r917u29n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_78(8171729) in pi_ohern on r917u29n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_79(8171730) in pi_ohern on r917u29n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_80(8171731) in pi_ohern on r917u29n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_81(8171732) in pi_ohern on r917u29n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_82(8171733) in pi_ohern on r815u03n11 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_83(8171734) in pi_ohern on r815u03n11 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_84(8171735) in pi_ohern on r815u03n11 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_85(8171736) in pi_ohern on r815u03n11 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_86(8171737) in pi_ohern on r815u03n11 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_87(8171738) in pi_ohern on r815u03n11 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_88(8171739) in pi_ohern on r815u03n12 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_89(8171740) in pi_ohern on r815u03n12 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_90(8171741) in pi_ohern on r911u32n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_91(8171742) in pi_ohern on r911u32n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_92(8171743) in pi_ohern on r807u22n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_93(8171744) in pi_ohern on r807u22n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_94(8171745) in pi_ohern on r807u22n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_95(8171746) in pi_ohern on r807u22n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_96(8171747) in pi_ohern on r807u24n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_97(8171748) in pi_ohern on r903u12n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_98(8171749) in pi_ohern on r903u24n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_99(8171750) in pi_ohern on r913u32n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_100(8171751) in pi_ohern on r913u32n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_101(8171752) in pi_ohern on r913u32n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_102(8171753) in pi_ohern on r913u35n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8171692 in day on r919u05n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8171691 in gpu on r816u35n07 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_699(8171754) in day on r919u05n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_700(8171755) in day on r806u09n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_701(8171756) in day on r806u09n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_702(8171757) in day on r806u11n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_703(8171758) in day on r806u11n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_704(8171759) in day on r816u09n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_705(8171760) in day on r816u09n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_706(8171761) in day on r816u09n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_707(8171762) in day on r816u09n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_708(8171763) in day on r816u09n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_709(8171764) in day on r816u09n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_710(8171765) in day on r816u09n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_711(8171766) in day on r816u09n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_712(8171767) in day on r816u09n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_713(8171768) in day on r816u09n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_714(8171769) in day on r816u09n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_715(8171770) in day on r816u09n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_716(8171771) in day on r816u09n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_717(8171772) in day on r816u09n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_718(8171773) in day on r816u09n06 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_719(8171774) in day on r816u09n06 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_720(8171775) in day on r816u09n06 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_721(8171776) in day on r816u09n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_722(8171777) in day on r816u09n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_723(8171778) in day on r816u15n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_724(8171779) in day on r816u15n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_725(8171780) in day on r816u15n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_726(8171781) in day on r816u15n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_727(8171782) in day on r816u15n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_728(8171783) in day on r816u15n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_729(8171784) in day on r816u15n02 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_730(8171785) in day on r816u15n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_731(8171786) in day on r816u15n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_732(8171787) in day on r816u15n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_733(8171788) in day on r816u15n03 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_734(8171789) in day on r816u15n04 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_735(8171790) in day on r816u15n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_736(8171791) in day on r816u15n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_737(8171792) in day on r816u15n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_738(8171793) in day on r816u15n05 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_739(8171794) in day on r816u15n06 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_740(8171795) in day on r816u15n06 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_741(8171796) in day on r816u15n06 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_742(8171797) in day on r816u15n06 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_743(8171798) in day on r816u15n07 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_744(8171799) in day on r816u15n07 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_745(8171800) in day on r816u15n07 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_746(8171801) in day on r816u15n07 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_747(8171802) in day on r816u15n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_748(8171803) in day on r816u15n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_749(8171804) in day on r816u15n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_750(8171805) in day on r816u15n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_751(8171806) in day on r816u15n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_752(8171807) in day on r816u15n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_753(8171808) in day on r816u15n08 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3994(8171809) in gpu on r808u14n01 Nov 05 11:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3995(8171810) in gpu on r903u34n01 Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_20(8170014) WEXITSTATUS 0 Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_20(8170014) done Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_109(8170120) WEXITSTATUS 0 Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_109(8170120) done Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_133(8170144) WEXITSTATUS 0 Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_133(8170144) done Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_181(8170192) WEXITSTATUS 0 Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_181(8170192) done Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_179(8170190) WEXITSTATUS 0 Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_179(8170190) done Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_20(7570983) WEXITSTATUS 0 Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_20(7570983) done Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1065(7570727) WEXITSTATUS 0 Nov 05 11:01:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1065(7570727) done Nov 05 11:01:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_356(8170508) WEXITSTATUS 0 Nov 05 11:01:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_356(8170508) done Nov 05 11:01:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7832013_0(7832013) in scavenge_gpu on r902u07n01 Nov 05 11:01:49 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 11:01:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7702(8168497) WEXITSTATUS 1 Nov 05 11:01:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7702(8168497) done Nov 05 11:01:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_15(8159260) WEXITSTATUS 0 Nov 05 11:01:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8158658_15(8159260) done Nov 05 11:01:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171688 NodeList=r816u35n03 #CPUs=8 Partition=week Nov 05 11:01:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7829(8169522) WEXITSTATUS 1 Nov 05 11:01:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7829(8169522) done Nov 05 11:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7856(8169625) WEXITSTATUS 1 Nov 05 11:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7856(8169625) done Nov 05 11:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_290(8170427) WEXITSTATUS 0 Nov 05 11:01:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_290(8170427) done Nov 05 11:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_153(8170164) WEXITSTATUS 0 Nov 05 11:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_153(8170164) done Nov 05 11:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_283(8170420) WEXITSTATUS 0 Nov 05 11:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_283(8170420) done Nov 05 11:01:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8010(8171811) in scavenge on r802u04n03 Nov 05 11:01:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8011(8171812) in scavenge on r802u06n03 Nov 05 11:01:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8012(8171813) in scavenge on r802u09n01 Nov 05 11:01:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8013(8171814) in scavenge on r802u11n02 Nov 05 11:01:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8014(8171815) in scavenge on r815u29n12 Nov 05 11:01:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8015(8171816) in scavenge on r903u34n01 Nov 05 11:01:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8016(8171817) in scavenge on r907u34n01 Nov 05 11:01:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8017(8171818) in scavenge on r807u37n01 Nov 05 11:01:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8171688 uid 25787 Nov 05 11:01:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_234(8170340) WEXITSTATUS 0 Nov 05 11:01:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_234(8170340) done Nov 05 11:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_155(8170166) WEXITSTATUS 0 Nov 05 11:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_155(8170166) done Nov 05 11:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_140(8170151) WEXITSTATUS 0 Nov 05 11:02:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_140(8170151) done Nov 05 11:02:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_40(8170034) WEXITSTATUS 0 Nov 05 11:02:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_40(8170034) done Nov 05 11:02:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_44(8170038) WEXITSTATUS 0 Nov 05 11:02:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_44(8170038) done Nov 05 11:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_122(8170133) WEXITSTATUS 0 Nov 05 11:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_122(8170133) done Nov 05 11:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_222(8170328) WEXITSTATUS 0 Nov 05 11:02:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_222(8170328) done Nov 05 11:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_870(8127984) WEXITSTATUS 0 Nov 05 11:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_870(8127984) done Nov 05 11:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_141(8170152) WEXITSTATUS 0 Nov 05 11:02:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_141(8170152) done Nov 05 11:02:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_177(8170188) WEXITSTATUS 0 Nov 05 11:02:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_177(8170188) done Nov 05 11:02:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_146(8170157) WEXITSTATUS 0 Nov 05 11:02:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_146(8170157) done Nov 05 11:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_185(8170291) WEXITSTATUS 0 Nov 05 11:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_185(8170291) done Nov 05 11:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_350(8170502) WEXITSTATUS 0 Nov 05 11:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_350(8170502) done Nov 05 11:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_231(8170337) WEXITSTATUS 0 Nov 05 11:02:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_231(8170337) done Nov 05 11:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_344(8170496) WEXITSTATUS 0 Nov 05 11:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_344(8170496) done Nov 05 11:02:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_125(8170136) WEXITSTATUS 0 Nov 05 11:02:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_125(8170136) done Nov 05 11:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_298(8170435) WEXITSTATUS 0 Nov 05 11:02:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_298(8170435) done Nov 05 11:02:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7997(8171513) WEXITSTATUS 1 Nov 05 11:02:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7997(8171513) done Nov 05 11:02:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171819 InitPrio=27319 usec=3239 Nov 05 11:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_237(8170343) WEXITSTATUS 0 Nov 05 11:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_237(8170343) done Nov 05 11:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_121(8170132) WEXITSTATUS 0 Nov 05 11:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_121(8170132) done Nov 05 11:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_49(8170043) WEXITSTATUS 0 Nov 05 11:02:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_49(8170043) done Nov 05 11:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_228(8170334) WEXITSTATUS 0 Nov 05 11:02:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_228(8170334) done Nov 05 11:02:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_22(8170016) WEXITSTATUS 0 Nov 05 11:02:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_22(8170016) done Nov 05 11:02:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_105(8170116) WEXITSTATUS 0 Nov 05 11:02:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_105(8170116) done Nov 05 11:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_149(8170160) WEXITSTATUS 0 Nov 05 11:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_149(8170160) done Nov 05 11:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170288 WEXITSTATUS 0 Nov 05 11:02:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170288 done Nov 05 11:02:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_357(8170509) WEXITSTATUS 0 Nov 05 11:02:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_357(8170509) done Nov 05 11:02:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_106(8170117) WEXITSTATUS 0 Nov 05 11:02:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_106(8170117) done Nov 05 11:02:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8167019 uid 21916 Nov 05 11:02:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_232(8170338) WEXITSTATUS 0 Nov 05 11:02:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_232(8170338) done Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171819 NodeList=r918u01n02 #CPUs=4 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171121 NodeList=r816u09n05 #CPUs=24 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_754(8171820) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_755(8171821) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_756(8171822) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_757(8171823) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_758(8171824) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_759(8171825) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_760(8171826) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_761(8171827) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_762(8171828) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_763(8171829) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_764(8171830) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_765(8171831) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_766(8171832) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_767(8171833) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_768(8171834) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_769(8171835) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_770(8171836) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_771(8171837) NodeList=r918u01n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_772(8171838) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_773(8171839) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_774(8171840) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_775(8171841) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_776(8171842) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_777(8171843) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_778(8171844) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_779(8171845) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_780(8171846) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_781(8171847) NodeList=r919u36n04 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_782(8171848) NodeList=r816u09n05 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_783(8171849) NodeList=r816u09n05 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_784(8171850) NodeList=r816u09n05 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_785(8171851) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 11:02:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_786(8171852) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 11:02:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_41(7571005) WEXITSTATUS 0 Nov 05 11:02:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_41(7571005) done Nov 05 11:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_352(8170504) WEXITSTATUS 0 Nov 05 11:02:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_352(8170504) done Nov 05 11:02:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_114(8170125) WEXITSTATUS 0 Nov 05 11:02:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_114(8170125) done Nov 05 11:02:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7986(8171301) WEXITSTATUS 1 Nov 05 11:02:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7986(8171301) done Nov 05 11:02:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_358(8170510) WEXITSTATUS 0 Nov 05 11:02:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_358(8170510) done Nov 05 11:02:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_119(8170130) WEXITSTATUS 0 Nov 05 11:02:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_119(8170130) done Nov 05 11:02:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_135(8170146) WEXITSTATUS 0 Nov 05 11:02:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_135(8170146) done Nov 05 11:03:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_126(8170137) WEXITSTATUS 0 Nov 05 11:03:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_126(8170137) done Nov 05 11:03:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_285(8170422) WEXITSTATUS 0 Nov 05 11:03:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_285(8170422) done Nov 05 11:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_288(8170425) WEXITSTATUS 0 Nov 05 11:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_288(8170425) done Nov 05 11:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_113(8170124) WEXITSTATUS 0 Nov 05 11:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_113(8170124) done Nov 05 11:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_346(8170498) WEXITSTATUS 0 Nov 05 11:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_346(8170498) done Nov 05 11:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_143(8170154) WEXITSTATUS 0 Nov 05 11:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_143(8170154) done Nov 05 11:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_142(8170153) WEXITSTATUS 0 Nov 05 11:03:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_142(8170153) done Nov 05 11:03:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_229(8170335) WEXITSTATUS 0 Nov 05 11:03:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_229(8170335) done Nov 05 11:03:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_233(8170339) WEXITSTATUS 0 Nov 05 11:03:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_233(8170339) done Nov 05 11:03:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_236(8170342) WEXITSTATUS 0 Nov 05 11:03:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_236(8170342) done Nov 05 11:03:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_104(8170115) WEXITSTATUS 0 Nov 05 11:03:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_104(8170115) done Nov 05 11:03:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3213(8132439) Nov 05 11:03:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_238(8170344) WEXITSTATUS 0 Nov 05 11:03:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_238(8170344) done Nov 05 11:03:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_199(8170305) WEXITSTATUS 0 Nov 05 11:03:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_199(8170305) done Nov 05 11:03:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_151(8170162) WEXITSTATUS 0 Nov 05 11:03:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_151(8170162) done Nov 05 11:03:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_148(8170159) WEXITSTATUS 0 Nov 05 11:03:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_148(8170159) done Nov 05 11:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_108(8170119) WEXITSTATUS 0 Nov 05 11:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_108(8170119) done Nov 05 11:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_170(8170181) WEXITSTATUS 0 Nov 05 11:03:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_170(8170181) done Nov 05 11:03:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 11:03:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 11:03:20 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_301(8170438) WEXITSTATUS 0 Nov 05 11:03:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_301(8170438) done Nov 05 11:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_124(8170135) WEXITSTATUS 0 Nov 05 11:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_124(8170135) done Nov 05 11:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_154(8170165) WEXITSTATUS 0 Nov 05 11:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_154(8170165) done Nov 05 11:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_351(8170503) WEXITSTATUS 0 Nov 05 11:03:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_351(8170503) done Nov 05 11:03:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_225(8170331) WEXITSTATUS 0 Nov 05 11:03:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_225(8170331) done Nov 05 11:03:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_127(8170138) WEXITSTATUS 0 Nov 05 11:03:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_127(8170138) done Nov 05 11:03:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7796(8169273) WEXITSTATUS 1 Nov 05 11:03:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7796(8169273) done Nov 05 11:03:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_111(8170122) WEXITSTATUS 0 Nov 05 11:03:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_111(8170122) done Nov 05 11:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_297(8170434) WEXITSTATUS 0 Nov 05 11:03:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_297(8170434) done Nov 05 11:03:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_279(8170416) WEXITSTATUS 0 Nov 05 11:03:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_279(8170416) done Nov 05 11:03:40 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8171853 NodeList=(null) usec=55101 Nov 05 11:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_131(8170142) WEXITSTATUS 0 Nov 05 11:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_131(8170142) done Nov 05 11:03:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171853 NodeList=r908u33n01 #CPUs=4 Partition=admintest Nov 05 11:03:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_130(8170141) WEXITSTATUS 0 Nov 05 11:03:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_130(8170141) done Nov 05 11:03:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_235(8170341) WEXITSTATUS 0 Nov 05 11:03:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_235(8170341) done Nov 05 11:03:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8014(8171815) WEXITSTATUS 1 Nov 05 11:03:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8014(8171815) done Nov 05 11:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_353(8170505) WEXITSTATUS 0 Nov 05 11:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_353(8170505) done Nov 05 11:03:42 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 11:03:43 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 11:03:44 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 11:03:45 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 11:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_107(8170118) WEXITSTATUS 0 Nov 05 11:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_107(8170118) done Nov 05 11:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_176(8170187) WEXITSTATUS 0 Nov 05 11:03:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_176(8170187) done Nov 05 11:03:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_128(8170139) WEXITSTATUS 0 Nov 05 11:03:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_128(8170139) done Nov 05 11:03:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2832(8128449) WEXITSTATUS 1 Nov 05 11:03:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2832(8128449) done Nov 05 11:03:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_156(8170167) WEXITSTATUS 0 Nov 05 11:03:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_156(8170167) done Nov 05 11:03:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_282(8170419) WEXITSTATUS 0 Nov 05 11:03:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_282(8170419) done Nov 05 11:03:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_296(8170433) WEXITSTATUS 0 Nov 05 11:03:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_296(8170433) done Nov 05 11:03:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_110(8170121) WEXITSTATUS 0 Nov 05 11:03:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_110(8170121) done Nov 05 11:03:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_205(8170311) WEXITSTATUS 0 Nov 05 11:03:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_205(8170311) done Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_787(8171854) NodeList=r806u11n01 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_788(8171855) NodeList=r914u01n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_789(8171856) NodeList=r914u01n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_790(8171857) NodeList=r914u01n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_791(8171858) NodeList=r914u01n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_792(8171859) NodeList=r914u01n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_793(8171860) NodeList=r914u01n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_794(8171861) NodeList=r919u05n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_795(8171862) NodeList=r919u05n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_796(8171863) NodeList=r919u05n04 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_797(8171864) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_798(8171865) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_799(8171866) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_800(8171867) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_801(8171868) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_802(8171869) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_803(8171870) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_804(8171871) NodeList=r815u09n06 #CPUs=1 Partition=day Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_79(7571043) WEXITSTATUS 0 Nov 05 11:03:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_79(7571043) done Nov 05 11:03:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_214(8170320) WEXITSTATUS 0 Nov 05 11:03:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_214(8170320) done Nov 05 11:03:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_364(8170536) WEXITSTATUS 0 Nov 05 11:03:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_364(8170536) done Nov 05 11:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_280(8170417) WEXITSTATUS 0 Nov 05 11:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_280(8170417) done Nov 05 11:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_304(8170441) WEXITSTATUS 0 Nov 05 11:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_304(8170441) done Nov 05 11:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_120(8170131) WEXITSTATUS 0 Nov 05 11:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_120(8170131) done Nov 05 11:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_256(8170393) WEXITSTATUS 0 Nov 05 11:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_256(8170393) done Nov 05 11:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_287(8170424) WEXITSTATUS 0 Nov 05 11:04:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_287(8170424) done Nov 05 11:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_115(8170126) WEXITSTATUS 0 Nov 05 11:04:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_115(8170126) done Nov 05 11:04:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_294(8170431) WEXITSTATUS 0 Nov 05 11:04:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_294(8170431) done Nov 05 11:04:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_123(8170134) WEXITSTATUS 0 Nov 05 11:04:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_123(8170134) done Nov 05 11:04:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_161(8170172) WEXITSTATUS 0 Nov 05 11:04:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_161(8170172) done Nov 05 11:04:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8171872 InitPrio=39310 usec=3547 Nov 05 11:04:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_345(8170497) WEXITSTATUS 0 Nov 05 11:04:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_345(8170497) done Nov 05 11:04:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_284(8170421) WEXITSTATUS 0 Nov 05 11:04:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_284(8170421) done Nov 05 11:04:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_118(8170129) WEXITSTATUS 0 Nov 05 11:04:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_118(8170129) done Nov 05 11:04:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_292(8170429) WEXITSTATUS 0 Nov 05 11:04:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_292(8170429) done Nov 05 11:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_144(8170155) WEXITSTATUS 0 Nov 05 11:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_144(8170155) done Nov 05 11:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_112(8170123) WEXITSTATUS 0 Nov 05 11:04:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_112(8170123) done Nov 05 11:04:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_289(8170426) WEXITSTATUS 0 Nov 05 11:04:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_289(8170426) done Nov 05 11:04:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_207(8170313) WEXITSTATUS 0 Nov 05 11:04:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_207(8170313) done Nov 05 11:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_163(8170174) WEXITSTATUS 0 Nov 05 11:04:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_163(8170174) done Nov 05 11:04:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_116(8170127) WEXITSTATUS 0 Nov 05 11:04:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_116(8170127) done Nov 05 11:04:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_147(8170158) WEXITSTATUS 0 Nov 05 11:04:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_147(8170158) done Nov 05 11:04:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_138(8170149) WEXITSTATUS 0 Nov 05 11:04:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_138(8170149) done Nov 05 11:04:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_220(8170326) WEXITSTATUS 0 Nov 05 11:04:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_220(8170326) done Nov 05 11:04:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_23(8170017) WEXITSTATUS 0 Nov 05 11:04:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_23(8170017) done Nov 05 11:04:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_281(8170418) WEXITSTATUS 0 Nov 05 11:04:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_281(8170418) done Nov 05 11:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_203(8170309) WEXITSTATUS 0 Nov 05 11:04:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_203(8170309) done Nov 05 11:04:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_219(8170325) WEXITSTATUS 0 Nov 05 11:04:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_219(8170325) done Nov 05 11:04:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_3(8169209) WEXITSTATUS 0 Nov 05 11:04:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169126_3(8169209) done Nov 05 11:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_359(8170511) WEXITSTATUS 0 Nov 05 11:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_359(8170511) done Nov 05 11:04:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_150(8170161) WEXITSTATUS 0 Nov 05 11:04:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_150(8170161) done Nov 05 11:04:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_308(8170445) WEXITSTATUS 0 Nov 05 11:04:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_308(8170445) done Nov 05 11:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_136(8170147) WEXITSTATUS 0 Nov 05 11:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_136(8170147) done Nov 05 11:04:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_202(8170308) WEXITSTATUS 0 Nov 05 11:04:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_202(8170308) done Nov 05 11:04:44 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1500 Nov 05 11:04:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_160(8170171) WEXITSTATUS 0 Nov 05 11:04:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_160(8170171) done Nov 05 11:04:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5553(8161043) WEXITSTATUS 1 Nov 05 11:04:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5553(8161043) done Nov 05 11:04:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_303(8170440) WEXITSTATUS 0 Nov 05 11:04:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_303(8170440) done Nov 05 11:04:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_230(8170336) WEXITSTATUS 0 Nov 05 11:04:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_230(8170336) done Nov 05 11:04:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_293(8170430) WEXITSTATUS 0 Nov 05 11:04:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_293(8170430) done Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_103(8171873) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_104(8171874) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_105(8171875) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_106(8171876) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_107(8171877) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_108(8171878) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_109(8171879) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_110(8171880) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_111(8171881) in pi_ohern on r801u32n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_112(8171882) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_113(8171883) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_114(8171884) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_115(8171885) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_116(8171886) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_117(8171887) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_118(8171888) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_119(8171889) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_120(8171890) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_121(8171891) in pi_ohern on r917u13n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_122(8171892) in pi_ohern on r917u13n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_123(8171893) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_124(8171894) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_125(8171895) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_126(8171896) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_127(8171897) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_128(8171898) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_129(8171899) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_130(8171900) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_131(8171901) in pi_ohern on r917u13n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_132(8171902) in pi_ohern on r917u13n04 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_133(8171903) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_134(8171904) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_135(8171905) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_136(8171906) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_137(8171907) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_138(8171908) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_139(8171909) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_140(8171910) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_141(8171911) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_142(8171912) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_143(8171913) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_144(8171914) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_145(8171915) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_146(8171916) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_147(8171917) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_148(8171918) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_149(8171919) in pi_ohern on r917u29n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_150(8171920) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_151(8171921) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_152(8171922) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_153(8171923) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_154(8171924) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_155(8171925) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_156(8171926) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_157(8171927) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_158(8171928) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_159(8171929) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_160(8171930) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_161(8171931) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_162(8171932) in pi_ohern on r917u29n02 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_163(8171933) in pi_ohern on r917u29n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_164(8171934) in pi_ohern on r917u29n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_165(8171935) in pi_ohern on r917u29n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_166(8171936) in pi_ohern on r917u29n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_167(8171937) in pi_ohern on r917u29n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_168(8171938) in pi_ohern on r917u29n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_169(8171939) in pi_ohern on r917u29n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_170(8171940) in pi_ohern on r917u29n03 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_171(8171941) in pi_ohern on r917u29n04 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_172(8171942) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_173(8171943) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_174(8171944) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_175(8171945) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_176(8171946) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_177(8171947) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_178(8171948) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_179(8171949) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_180(8171950) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_181(8171951) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_182(8171952) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_183(8171953) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_184(8171954) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_185(8171955) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_186(8171956) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_187(8171957) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_188(8171958) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_189(8171959) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_190(8171960) in pi_ohern on r815u03n11 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_191(8171961) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_192(8171962) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_193(8171963) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_194(8171964) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_195(8171965) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_196(8171966) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_197(8171967) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_198(8171968) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_199(8171969) in pi_ohern on r815u03n12 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_200(8171970) in pi_ohern on r903u24n01 Nov 05 11:04:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_201(8171971) in pi_ohern on r903u24n01 Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_157(8170168) WEXITSTATUS 0 Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_157(8170168) done Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171872 NodeList=r806u33n01 #CPUs=2 Partition=gpu_devel Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_805(8171972) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_806(8171973) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_807(8171974) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_808(8171975) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_809(8171976) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_810(8171977) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_811(8171978) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_812(8171979) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_813(8171980) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_814(8171981) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_815(8171982) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_816(8171983) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_817(8171984) NodeList=r816u15n03 #CPUs=1 Partition=day Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_226(8170332) WEXITSTATUS 0 Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_226(8170332) done Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_197(8170303) WEXITSTATUS 0 Nov 05 11:04:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_197(8170303) done Nov 05 11:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_109(7571073) WEXITSTATUS 0 Nov 05 11:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_109(7571073) done Nov 05 11:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_158(8170169) WEXITSTATUS 0 Nov 05 11:04:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_158(8170169) done Nov 05 11:04:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_145(8170156) WEXITSTATUS 0 Nov 05 11:04:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_145(8170156) done Nov 05 11:04:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3982(8170641) WEXITSTATUS 0 Nov 05 11:04:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3982(8170641) done Nov 05 11:04:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:04:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_187(8170293) WEXITSTATUS 0 Nov 05 11:04:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_187(8170293) done Nov 05 11:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_193(8170299) WEXITSTATUS 0 Nov 05 11:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_193(8170299) done Nov 05 11:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_162(8170173) WEXITSTATUS 0 Nov 05 11:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_162(8170173) done Nov 05 11:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3995(8171810) WEXITSTATUS 0 Nov 05 11:04:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3995(8171810) done Nov 05 11:05:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8018(8171985) in scavenge on r801u25n01 Nov 05 11:05:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8019(8171986) in scavenge on r802u14n01 Nov 05 11:05:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8020(8171987) in scavenge on r815u29n12 Nov 05 11:05:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8021(8171988) in scavenge on r816u29n11 Nov 05 11:05:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8022(8171989) in scavenge on r816u29n11 Nov 05 11:05:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8023(8171990) in scavenge on r808u17n01 Nov 05 11:05:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_237(8114503) in scavenge on r807u24n01 Nov 05 11:05:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_239(8114505) in scavenge on r816u35n03 Nov 05 11:05:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_306(8170443) WEXITSTATUS 0 Nov 05 11:05:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_306(8170443) done Nov 05 11:05:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_227(8170333) WEXITSTATUS 0 Nov 05 11:05:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_227(8170333) done Nov 05 11:05:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3221(8132556) Nov 05 11:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7685(8168328) WEXITSTATUS 1 Nov 05 11:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7685(8168328) done Nov 05 11:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_139(8170150) WEXITSTATUS 0 Nov 05 11:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_139(8170150) done Nov 05 11:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_339(8170491) WEXITSTATUS 0 Nov 05 11:05:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_339(8170491) done Nov 05 11:05:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_198(8170304) WEXITSTATUS 0 Nov 05 11:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_198(8170304) done Nov 05 11:05:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:17 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 11:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_381(8170671) WEXITSTATUS 0 Nov 05 11:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_381(8170671) done Nov 05 11:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6951(8165243) WEXITSTATUS 1 Nov 05 11:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6951(8165243) done Nov 05 11:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3994(8171809) WEXITSTATUS 0 Nov 05 11:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3994(8171809) done Nov 05 11:05:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171853 WEXITSTATUS 0 Nov 05 11:05:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171853 done Nov 05 11:05:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_194(8170300) WEXITSTATUS 0 Nov 05 11:05:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_194(8170300) done Nov 05 11:05:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8171991 NodeList=(null) usec=30093 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8171991 NodeList=r908u28n02 #CPUs=1 Partition=admintest Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_152(8170163) WEXITSTATUS 0 Nov 05 11:05:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_152(8170163) done Nov 05 11:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_7(8169222) WEXITSTATUS 0 Nov 05 11:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169129_7(8169222) done Nov 05 11:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_204(8170310) WEXITSTATUS 0 Nov 05 11:05:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_204(8170310) done Nov 05 11:05:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_217(8170323) WEXITSTATUS 0 Nov 05 11:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_217(8170323) done Nov 05 11:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_213(8170319) WEXITSTATUS 0 Nov 05 11:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_213(8170319) done Nov 05 11:05:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_132(8170143) WEXITSTATUS 0 Nov 05 11:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_132(8170143) done Nov 05 11:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_189(8170295) WEXITSTATUS 0 Nov 05 11:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_189(8170295) done Nov 05 11:05:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_355(8170507) WEXITSTATUS 0 Nov 05 11:05:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_355(8170507) done Nov 05 11:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_201(8170307) WEXITSTATUS 0 Nov 05 11:05:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_201(8170307) done Nov 05 11:05:35 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 26553: QOSMaxSubmitJobPerUserLimit Nov 05 11:05:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:05:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_184(8170290) WEXITSTATUS 0 Nov 05 11:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_184(8170290) done Nov 05 11:05:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_190(8170296) WEXITSTATUS 0 Nov 05 11:05:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_190(8170296) done Nov 05 11:05:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_218(8170324) WEXITSTATUS 0 Nov 05 11:05:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_218(8170324) done Nov 05 11:05:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_305(8170442) WEXITSTATUS 0 Nov 05 11:05:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_305(8170442) done Nov 05 11:05:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7410(8166189) WEXITSTATUS 1 Nov 05 11:05:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7410(8166189) done Nov 05 11:05:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_302(8170439) WEXITSTATUS 0 Nov 05 11:05:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_302(8170439) done Nov 05 11:05:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_348(8170500) WEXITSTATUS 0 Nov 05 11:05:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_348(8170500) done Nov 05 11:05:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7904(8170203) WEXITSTATUS 1 Nov 05 11:05:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7904(8170203) done Nov 05 11:05:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_261(8170398) WEXITSTATUS 0 Nov 05 11:05:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_261(8170398) done Nov 05 11:05:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_186(8170292) WEXITSTATUS 0 Nov 05 11:05:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_186(8170292) done Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_818(8171992) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_819(8171993) NodeList=r918u01n01 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_820(8171994) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_821(8171995) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_822(8171996) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_823(8171997) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_824(8171998) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_825(8171999) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_826(8172000) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_827(8172001) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_828(8172002) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_829(8172003) NodeList=r815u35n05 #CPUs=1 Partition=day Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_271(7571236) WEXITSTATUS 0 Nov 05 11:05:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_271(7571236) done Nov 05 11:05:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_335(8170487) WEXITSTATUS 0 Nov 05 11:05:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_335(8170487) done Nov 05 11:05:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6924(8165196) WEXITSTATUS 1 Nov 05 11:05:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6924(8165196) done Nov 05 11:05:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:05:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_196(8170302) WEXITSTATUS 0 Nov 05 11:05:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_196(8170302) done Nov 05 11:06:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8171819 uid 25787 Nov 05 11:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3973(8170194) WEXITSTATUS 0 Nov 05 11:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3973(8170194) done Nov 05 11:06:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_336(8170488) WEXITSTATUS 0 Nov 05 11:06:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_336(8170488) done Nov 05 11:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_338(8170490) WEXITSTATUS 0 Nov 05 11:06:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_338(8170490) done Nov 05 11:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_263(8170400) WEXITSTATUS 0 Nov 05 11:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_263(8170400) done Nov 05 11:06:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_377(8170667) WEXITSTATUS 0 Nov 05 11:06:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_377(8170667) done Nov 05 11:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_129(8170140) WEXITSTATUS 0 Nov 05 11:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_129(8170140) done Nov 05 11:06:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_300(8170437) WEXITSTATUS 0 Nov 05 11:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_300(8170437) done Nov 05 11:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_272(8170409) WEXITSTATUS 0 Nov 05 11:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_272(8170409) done Nov 05 11:06:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_117(8170128) WEXITSTATUS 0 Nov 05 11:06:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_117(8170128) done Nov 05 11:06:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_386(8170676) WEXITSTATUS 0 Nov 05 11:06:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_386(8170676) done Nov 05 11:06:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_215(8170321) WEXITSTATUS 0 Nov 05 11:06:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_215(8170321) done Nov 05 11:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_243(8170380) WEXITSTATUS 0 Nov 05 11:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_243(8170380) done Nov 05 11:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_195(8170301) WEXITSTATUS 0 Nov 05 11:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_195(8170301) done Nov 05 11:06:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_211(8170317) WEXITSTATUS 0 Nov 05 11:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_211(8170317) done Nov 05 11:06:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_340(8170492) WEXITSTATUS 0 Nov 05 11:06:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_340(8170492) done Nov 05 11:06:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_159(8170170) WEXITSTATUS 0 Nov 05 11:06:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_159(8170170) done Nov 05 11:06:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_363(8170535) WEXITSTATUS 0 Nov 05 11:06:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_363(8170535) done Nov 05 11:06:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_376(8170666) WEXITSTATUS 0 Nov 05 11:06:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_376(8170666) done Nov 05 11:06:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_208(8170314) WEXITSTATUS 0 Nov 05 11:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_208(8170314) done Nov 05 11:06:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_137(8170148) WEXITSTATUS 0 Nov 05 11:06:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_137(8170148) done Nov 05 11:06:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_337(8170489) WEXITSTATUS 0 Nov 05 11:06:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_337(8170489) done Nov 05 11:06:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_379(8170669) WEXITSTATUS 0 Nov 05 11:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_379(8170669) done Nov 05 11:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_318(8170470) WEXITSTATUS 0 Nov 05 11:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_318(8170470) done Nov 05 11:06:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7848334 uid 22864 Nov 05 11:06:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_307(8170444) WEXITSTATUS 0 Nov 05 11:06:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_307(8170444) done Nov 05 11:06:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7951(8170652) WEXITSTATUS 1 Nov 05 11:06:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7951(8170652) done Nov 05 11:06:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7999(8171673) WEXITSTATUS 1 Nov 05 11:06:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7999(8171673) done Nov 05 11:06:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_254(8170391) WEXITSTATUS 0 Nov 05 11:06:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_254(8170391) done Nov 05 11:06:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_242(8170379) WEXITSTATUS 0 Nov 05 11:06:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_242(8170379) done Nov 05 11:06:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_299(8170436) WEXITSTATUS 0 Nov 05 11:06:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_299(8170436) done Nov 05 11:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_380(8170670) WEXITSTATUS 0 Nov 05 11:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_380(8170670) done Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_830(8172004) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_831(8172005) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_832(8172006) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_833(8172007) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_834(8172008) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_835(8172009) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_836(8172010) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_837(8172011) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_838(8172012) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_839(8172013) NodeList=r807u05n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_840(8172014) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_841(8172015) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_842(8172016) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_843(8172017) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_844(8172018) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_845(8172019) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_846(8172020) NodeList=r918u01n02 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_847(8172021) NodeList=r918u09n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_848(8172022) NodeList=r918u09n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_849(8172023) NodeList=r918u09n04 #CPUs=1 Partition=day Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_318(7571283) WEXITSTATUS 0 Nov 05 11:06:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_318(7571283) done Nov 05 11:06:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_192(8170298) WEXITSTATUS 0 Nov 05 11:06:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_192(8170298) done Nov 05 11:06:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_385(8170675) WEXITSTATUS 0 Nov 05 11:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_385(8170675) done Nov 05 11:07:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_250(8170387) WEXITSTATUS 0 Nov 05 11:07:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_250(8170387) done Nov 05 11:07:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_183(8170289) WEXITSTATUS 0 Nov 05 11:07:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_183(8170289) done Nov 05 11:07:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:12 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651306_3(7651391) Nov 05 11:07:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_275(8170412) WEXITSTATUS 0 Nov 05 11:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_275(8170412) done Nov 05 11:07:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_367(8170539) WEXITSTATUS 0 Nov 05 11:07:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_367(8170539) done Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:07:24 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:07:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_366(8170538) WEXITSTATUS 0 Nov 05 11:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_366(8170538) done Nov 05 11:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_248(8170385) WEXITSTATUS 0 Nov 05 11:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_248(8170385) done Nov 05 11:07:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_200(8170306) WEXITSTATUS 0 Nov 05 11:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_200(8170306) done Nov 05 11:07:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2596(8166233) has been requeued to reclaim resources for JobId=8169916 Nov 05 11:07:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7787(8169264) has been requeued to reclaim resources for JobId=8169916 Nov 05 11:07:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3678(8138254) has been requeued to reclaim resources for JobId=8169916 Nov 05 11:07:32 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2374(8165581) has been requeued to reclaim resources for JobId=8169916 Nov 05 11:07:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7696(8168469) WEXITSTATUS 1 Nov 05 11:07:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7696(8168469) done Nov 05 11:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7601(8167030) WEXITSTATUS 1 Nov 05 11:07:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7601(8167030) done Nov 05 11:07:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7787(8169264) Nov 05 11:07:34 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3678(8138254) Nov 05 11:07:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2596(8166233) Nov 05 11:07:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2374(8165581) Nov 05 11:07:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8169916 NodeList=r815u29n08 #CPUs=48 Partition=pi_mingarelli Nov 05 11:07:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7165(8165783) WEXITSTATUS 1 Nov 05 11:07:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7165(8165783) done Nov 05 11:07:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_257(8170394) WEXITSTATUS 0 Nov 05 11:07:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_257(8170394) done Nov 05 11:07:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_295(8170432) WEXITSTATUS 0 Nov 05 11:07:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_295(8170432) done Nov 05 11:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_321(8170473) WEXITSTATUS 0 Nov 05 11:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_321(8170473) done Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_202(8172024) in pi_ohern on r801u32n02 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_203(8172025) in pi_ohern on r917u13n01 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_204(8172026) in pi_ohern on r917u13n01 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_205(8172027) in pi_ohern on r917u13n01 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_206(8172028) in pi_ohern on r917u13n01 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_207(8172029) in pi_ohern on r917u13n01 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_208(8172030) in pi_ohern on r917u13n01 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_209(8172031) in pi_ohern on r917u13n01 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_210(8172032) in pi_ohern on r917u13n02 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_211(8172033) in pi_ohern on r917u13n02 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_212(8172034) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_213(8172035) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_214(8172036) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_215(8172037) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_216(8172038) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_217(8172039) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_218(8172040) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_219(8172041) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_220(8172042) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_221(8172043) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_222(8172044) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_223(8172045) in pi_ohern on r917u13n03 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_224(8172046) in pi_ohern on r917u13n04 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_225(8172047) in pi_ohern on r917u13n04 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_226(8172048) in pi_ohern on r917u13n04 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_227(8172049) in pi_ohern on r917u13n04 Nov 05 11:07:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_228(8172050) in pi_ohern on r917u29n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_229(8172051) in pi_ohern on r917u29n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_230(8172052) in pi_ohern on r917u29n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_231(8172053) in pi_ohern on r917u29n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_232(8172054) in pi_ohern on r917u29n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_233(8172055) in pi_ohern on r917u29n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_234(8172056) in pi_ohern on r917u29n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_235(8172057) in pi_ohern on r917u29n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_236(8172058) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_237(8172059) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_238(8172060) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_239(8172061) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_240(8172062) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_241(8172063) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_242(8172064) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_243(8172065) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_244(8172066) in pi_ohern on r917u29n02 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_245(8172067) in pi_ohern on r917u29n03 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_246(8172068) in pi_ohern on r917u29n03 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_247(8172069) in pi_ohern on r917u29n03 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_248(8172070) in pi_ohern on r917u29n03 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_249(8172071) in pi_ohern on r917u29n03 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_250(8172072) in pi_ohern on r917u29n03 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_251(8172073) in pi_ohern on r917u29n03 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_252(8172074) in pi_ohern on r917u29n03 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_253(8172075) in pi_ohern on r815u03n11 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_254(8172076) in pi_ohern on r815u03n11 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_255(8172077) in pi_ohern on r815u03n11 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_256(8172078) in pi_ohern on r815u03n11 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_257(8172079) in pi_ohern on r815u03n11 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_258(8172080) in pi_ohern on r815u03n11 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_259(8172081) in pi_ohern on r815u03n11 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_260(8172082) in pi_ohern on r815u03n12 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_261(8172083) in pi_ohern on r815u03n12 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_262(8172084) in pi_ohern on r815u03n12 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_263(8172085) in pi_ohern on r815u03n12 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_264(8172086) in pi_ohern on r815u03n12 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_265(8172087) in pi_ohern on r807u22n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_266(8172088) in pi_ohern on r807u22n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_267(8172089) in pi_ohern on r807u22n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_268(8172090) in pi_ohern on r807u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_269(8172091) in pi_ohern on r807u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_270(8172092) in pi_ohern on r807u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_271(8172093) in pi_ohern on r807u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_272(8172094) in pi_ohern on r807u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_273(8172095) in pi_ohern on r903u12n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_274(8172096) in pi_ohern on r903u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_275(8172097) in pi_ohern on r903u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_276(8172098) in pi_ohern on r903u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_277(8172099) in pi_ohern on r903u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_278(8172100) in pi_ohern on r903u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_279(8172101) in pi_ohern on r903u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_280(8172102) in pi_ohern on r903u24n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_281(8172103) in pi_ohern on r913u32n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_282(8172104) in pi_ohern on r913u32n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_283(8172105) in pi_ohern on r913u32n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_284(8172106) in pi_ohern on r913u32n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_285(8172107) in pi_ohern on r913u32n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_286(8172108) in pi_ohern on r913u35n01 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_206(8170312) WEXITSTATUS 0 Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_206(8170312) done Nov 05 11:07:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:07:48 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_850(8172109) in day on r805u11n01 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_851(8172110) in day on r805u11n01 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_852(8172111) in day on r913u24n03 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_853(8172112) in day on r913u24n03 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_854(8172113) in day on r913u24n03 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_855(8172114) in day on r913u24n03 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_237(8114503) WEXITSTATUS 0 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_237(8114503) done Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_209(8170315) WEXITSTATUS 0 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_209(8170315) done Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_212(8170318) WEXITSTATUS 0 Nov 05 11:07:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_212(8170318) done Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_856(8172115) in day on r913u24n03 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_857(8172116) in day on r913u24n03 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_858(8172117) in day on r913u24n03 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_859(8172118) in day on r913u24n03 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_860(8172119) in day on r913u24n03 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_861(8172120) in day on r913u24n03 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3996(8172121) in gpu on r903u34n01 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3997(8172122) in gpu on r808u08n01 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3998(8172123) in gpu on r808u14n01 Nov 05 11:07:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_3999(8172124) in gpu on r808u14n01 Nov 05 11:07:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_416(7571381) WEXITSTATUS 0 Nov 05 11:07:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_416(7571381) done Nov 05 11:07:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_276(8170413) WEXITSTATUS 0 Nov 05 11:07:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_276(8170413) done Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7327(8166024) WEXITSTATUS 1 Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7327(8166024) done Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_862(8172125) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_863(8172126) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_864(8172127) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_865(8172128) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_866(8172129) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_867(8172130) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_868(8172131) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_869(8172132) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_870(8172133) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_871(8172134) NodeList=r816u03n06 #CPUs=1 Partition=day Nov 05 11:07:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_548(7571513) WEXITSTATUS 0 Nov 05 11:07:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_548(7571513) done Nov 05 11:07:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_264(8170401) WEXITSTATUS 0 Nov 05 11:07:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_264(8170401) done Nov 05 11:08:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_249(8170386) WEXITSTATUS 0 Nov 05 11:08:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_249(8170386) done Nov 05 11:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_188(8170294) WEXITSTATUS 0 Nov 05 11:08:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_188(8170294) done Nov 05 11:08:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8024(8172135) in scavenge on r801u30n01 Nov 05 11:08:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8025(8172136) in scavenge on r911u09n01 Nov 05 11:08:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8026(8172137) in scavenge on r802u04n03 Nov 05 11:08:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8027(8172138) in scavenge on r802u04n04 Nov 05 11:08:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8028(8172139) in scavenge on r802u11n03 Nov 05 11:08:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8029(8172140) in scavenge on r816u23n07 Nov 05 11:08:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8030(8172141) in scavenge on r816u35n11 Nov 05 11:08:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8031(8172142) in scavenge on r907u28n01 Nov 05 11:08:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_393(8170683) WEXITSTATUS 0 Nov 05 11:08:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_393(8170683) done Nov 05 11:08:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_260(8170397) WEXITSTATUS 0 Nov 05 11:08:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_260(8170397) done Nov 05 11:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_252(8170389) WEXITSTATUS 0 Nov 05 11:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_252(8170389) done Nov 05 11:08:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_365(8170537) WEXITSTATUS 0 Nov 05 11:08:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_365(8170537) done Nov 05 11:08:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3996(8172121) WEXITSTATUS 0 Nov 05 11:08:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3996(8172121) done Nov 05 11:08:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_325(8170477) WEXITSTATUS 0 Nov 05 11:08:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_325(8170477) done Nov 05 11:08:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_389(8170679) WEXITSTATUS 0 Nov 05 11:08:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_389(8170679) done Nov 05 11:08:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8172143 NodeList=(null) usec=26229 Nov 05 11:08:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_984(7573153) WEXITSTATUS 0 Nov 05 11:08:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_984(7573153) done Nov 05 11:08:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_378(8170668) WEXITSTATUS 0 Nov 05 11:08:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_378(8170668) done Nov 05 11:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_360(8170532) WEXITSTATUS 0 Nov 05 11:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_360(8170532) done Nov 05 11:08:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172143 NodeList=r904u13n01 #CPUs=8 Partition=pi_ying_rex Nov 05 11:08:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_274(8170411) WEXITSTATUS 0 Nov 05 11:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_274(8170411) done Nov 05 11:08:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_259(8170396) WEXITSTATUS 0 Nov 05 11:08:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_259(8170396) done Nov 05 11:08:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_258(8170395) WEXITSTATUS 0 Nov 05 11:08:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_258(8170395) done Nov 05 11:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_314(8170466) WEXITSTATUS 0 Nov 05 11:08:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_314(8170466) done Nov 05 11:08:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_328(8170480) WEXITSTATUS 0 Nov 05 11:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_328(8170480) done Nov 05 11:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_255(8170392) WEXITSTATUS 0 Nov 05 11:08:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_255(8170392) done Nov 05 11:08:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_411(8170817) WEXITSTATUS 0 Nov 05 11:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_411(8170817) done Nov 05 11:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_273(8170410) WEXITSTATUS 0 Nov 05 11:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_273(8170410) done Nov 05 11:08:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_262(8170399) WEXITSTATUS 0 Nov 05 11:08:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_262(8170399) done Nov 05 11:08:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_271(8170408) WEXITSTATUS 0 Nov 05 11:08:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_271(8170408) done Nov 05 11:08:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3999(8172124) WEXITSTATUS 0 Nov 05 11:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3999(8172124) done Nov 05 11:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_382(8170672) WEXITSTATUS 0 Nov 05 11:08:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_382(8170672) done Nov 05 11:08:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_210(8170316) WEXITSTATUS 0 Nov 05 11:08:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_210(8170316) done Nov 05 11:08:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_319(8170471) WEXITSTATUS 0 Nov 05 11:08:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_319(8170471) done Nov 05 11:08:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172144 InitPrio=27319 usec=5274 Nov 05 11:08:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172144 NodeList=r805u09n02 #CPUs=4 Partition=day Nov 05 11:08:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_872(8172145) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 11:08:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_873(8172146) NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 11:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_559(7571524) WEXITSTATUS 0 Nov 05 11:08:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_559(7571524) done Nov 05 11:08:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_216(8170322) WEXITSTATUS 0 Nov 05 11:08:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_216(8170322) done Nov 05 11:08:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_244(8170381) WEXITSTATUS 0 Nov 05 11:08:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_244(8170381) done Nov 05 11:08:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_391(8170681) WEXITSTATUS 0 Nov 05 11:08:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_391(8170681) done Nov 05 11:08:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:08:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_315(8170467) WEXITSTATUS 0 Nov 05 11:08:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_315(8170467) done Nov 05 11:09:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_247(8170384) WEXITSTATUS 0 Nov 05 11:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_247(8170384) done Nov 05 11:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_251(8170388) WEXITSTATUS 0 Nov 05 11:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_251(8170388) done Nov 05 11:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_246(8170383) WEXITSTATUS 0 Nov 05 11:09:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_246(8170383) done Nov 05 11:09:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_311(8170463) WEXITSTATUS 0 Nov 05 11:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_311(8170463) done Nov 05 11:09:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172147 InitPrio=48211 usec=6347 Nov 05 11:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_265(8170402) WEXITSTATUS 0 Nov 05 11:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_265(8170402) done Nov 05 11:09:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_596(8114863) WEXITSTATUS 0 Nov 05 11:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_596(8114863) done Nov 05 11:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_654(7572823) WEXITSTATUS 0 Nov 05 11:09:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_654(7572823) done Nov 05 11:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_241(8170378) WEXITSTATUS 0 Nov 05 11:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_241(8170378) done Nov 05 11:09:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:10 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172147 NodeList=r806u37n01 #CPUs=16 Partition=gpu Nov 05 11:09:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3600(8136535) WEXITSTATUS 1 Nov 05 11:09:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3600(8136535) done Nov 05 11:09:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171691 WEXITSTATUS 0 Nov 05 11:09:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171691 done Nov 05 11:09:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7178(8165801) WEXITSTATUS 1 Nov 05 11:09:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7178(8165801) done Nov 05 11:09:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172148 InitPrio=24116 usec=7931 Nov 05 11:09:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172149 InitPrio=24116 usec=4277 Nov 05 11:09:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172150 InitPrio=24116 usec=3742 Nov 05 11:09:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172151 InitPrio=24116 usec=3593 Nov 05 11:09:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6849(8165049) WEXITSTATUS 1 Nov 05 11:09:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6849(8165049) done Nov 05 11:09:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_320(8170472) WEXITSTATUS 0 Nov 05 11:09:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_320(8170472) done Nov 05 11:09:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_191(8170297) WEXITSTATUS 0 Nov 05 11:09:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_191(8170297) done Nov 05 11:09:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:09:27 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6976(8165322) WEXITSTATUS 1 Nov 05 11:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6976(8165322) done Nov 05 11:09:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_341(8170493) WEXITSTATUS 0 Nov 05 11:09:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_341(8170493) done Nov 05 11:09:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_368(8170658) WEXITSTATUS 0 Nov 05 11:09:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_368(8170658) done Nov 05 11:09:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_240(8170377) WEXITSTATUS 0 Nov 05 11:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_240(8170377) done Nov 05 11:09:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172147 WEXITSTATUS 1 Nov 05 11:09:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172147 done Nov 05 11:09:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3998(8172123) WEXITSTATUS 0 Nov 05 11:09:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3998(8172123) done Nov 05 11:09:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_239(8170376) WEXITSTATUS 0 Nov 05 11:09:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_239(8170376) done Nov 05 11:09:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_407(8170813) WEXITSTATUS 0 Nov 05 11:09:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_407(8170813) done Nov 05 11:09:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172148 NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172149 NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172150 NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172151 NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_874(8172152) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_875(8172153) NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_876(8172154) NodeList=r913u05n01 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_877(8172155) NodeList=r913u05n01 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_878(8172156) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_879(8172157) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_880(8172158) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_881(8172159) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_882(8172160) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_883(8172161) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_884(8172162) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_885(8172163) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_886(8172164) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_887(8172165) NodeList=r816u09n02 #CPUs=1 Partition=day Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_591(7571556) WEXITSTATUS 0 Nov 05 11:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_591(7571556) done Nov 05 11:09:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_415(8170821) WEXITSTATUS 0 Nov 05 11:09:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_415(8170821) done Nov 05 11:09:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_270(8170407) WEXITSTATUS 0 Nov 05 11:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_270(8170407) done Nov 05 11:09:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7811(8169354) WEXITSTATUS 1 Nov 05 11:09:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7811(8169354) done Nov 05 11:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_278(8170415) WEXITSTATUS 0 Nov 05 11:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_278(8170415) done Nov 05 11:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_342(8170494) WEXITSTATUS 0 Nov 05 11:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_342(8170494) done Nov 05 11:10:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_245(8170382) WEXITSTATUS 0 Nov 05 11:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_245(8170382) done Nov 05 11:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_417(8170823) WEXITSTATUS 0 Nov 05 11:10:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_417(8170823) done Nov 05 11:10:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_312(8170464) WEXITSTATUS 0 Nov 05 11:10:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_312(8170464) done Nov 05 11:10:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_422(8170828) WEXITSTATUS 0 Nov 05 11:10:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_422(8170828) done Nov 05 11:10:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172144 uid 25787 Nov 05 11:10:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_430(8170836) WEXITSTATUS 0 Nov 05 11:10:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_430(8170836) done Nov 05 11:10:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172166 InitPrio=44396 usec=8591 Nov 05 11:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_253(8170390) WEXITSTATUS 0 Nov 05 11:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_253(8170390) done Nov 05 11:10:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172167 InitPrio=27319 usec=4375 Nov 05 11:10:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_317(8170469) WEXITSTATUS 0 Nov 05 11:10:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_317(8170469) done Nov 05 11:10:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_414(8170820) WEXITSTATUS 0 Nov 05 11:10:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_414(8170820) done Nov 05 11:10:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_327(8170479) WEXITSTATUS 0 Nov 05 11:10:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_327(8170479) done Nov 05 11:10:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_310(8170462) WEXITSTATUS 0 Nov 05 11:10:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_310(8170462) done Nov 05 11:10:27 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172168 InitPrio=48211 usec=3782 Nov 05 11:10:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172168 NodeList=r806u37n01 #CPUs=16 Partition=gpu Nov 05 11:10:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_326(8170478) WEXITSTATUS 0 Nov 05 11:10:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_326(8170478) done Nov 05 11:10:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_413(8170819) WEXITSTATUS 0 Nov 05 11:10:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_413(8170819) done Nov 05 11:10:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_267(8170404) WEXITSTATUS 0 Nov 05 11:10:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_267(8170404) done Nov 05 11:10:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5089(8158413) WEXITSTATUS 1 Nov 05 11:10:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5089(8158413) done Nov 05 11:10:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_362(8170534) WEXITSTATUS 0 Nov 05 11:10:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_362(8170534) done Nov 05 11:10:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_374(8170664) WEXITSTATUS 0 Nov 05 11:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_374(8170664) done Nov 05 11:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7683(8168326) WEXITSTATUS 1 Nov 05 11:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7683(8168326) done Nov 05 11:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172168 WEXITSTATUS 1 Nov 05 11:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172168 done Nov 05 11:10:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_316(8170468) WEXITSTATUS 0 Nov 05 11:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_316(8170468) done Nov 05 11:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 26553: QOSMaxSubmitJobPerUserLimit Nov 05 11:10:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:10:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_472(8170993) WEXITSTATUS 0 Nov 05 11:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_472(8170993) done Nov 05 11:10:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_269(8170406) WEXITSTATUS 0 Nov 05 11:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_269(8170406) done Nov 05 11:10:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172148 WEXITSTATUS 0 Nov 05 11:10:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172148 done Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_398(8170804) WEXITSTATUS 0 Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_398(8170804) done Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172168 uid 25227 Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172167 NodeList=r805u09n02 #CPUs=4 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_888(8172169) NodeList=r913u28n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_889(8172170) NodeList=r913u28n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_890(8172171) NodeList=r918u36n04 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_891(8172172) NodeList=r918u36n04 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_892(8172173) NodeList=r918u36n04 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_893(8172174) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_894(8172175) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_895(8172176) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_896(8172177) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_897(8172178) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_898(8172179) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_899(8172180) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_900(8172181) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_901(8172182) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_902(8172183) NodeList=r805u14n02 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_903(8172184) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_904(8172185) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_905(8172186) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:10:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_906(8172187) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:10:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_394(7571359) WEXITSTATUS 0 Nov 05 11:10:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_394(7571359) done Nov 05 11:10:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_639(7571604) WEXITSTATUS 0 Nov 05 11:10:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_639(7571604) done Nov 05 11:10:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_418(8170824) WEXITSTATUS 0 Nov 05 11:10:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_418(8170824) done Nov 05 11:10:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_287(8172188) in pi_ohern on r801u32n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_288(8172189) in pi_ohern on r801u32n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_289(8172190) in pi_ohern on r801u32n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_290(8172191) in pi_ohern on r801u32n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_291(8172192) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_292(8172193) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_293(8172194) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_294(8172195) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_295(8172196) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_296(8172197) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_297(8172198) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_298(8172199) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_299(8172200) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_300(8172201) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_301(8172202) in pi_ohern on r917u13n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_302(8172203) in pi_ohern on r917u13n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_303(8172204) in pi_ohern on r917u13n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_304(8172205) in pi_ohern on r917u13n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_305(8172206) in pi_ohern on r917u13n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_306(8172207) in pi_ohern on r917u13n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_307(8172208) in pi_ohern on r917u13n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_308(8172209) in pi_ohern on r917u13n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_309(8172210) in pi_ohern on r917u13n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_310(8172211) in pi_ohern on r917u13n03 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_311(8172212) in pi_ohern on r917u13n03 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_312(8172213) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_313(8172214) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_314(8172215) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_315(8172216) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_316(8172217) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_317(8172218) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_318(8172219) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_319(8172220) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_320(8172221) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_321(8172222) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_322(8172223) in pi_ohern on r917u13n04 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_323(8172224) in pi_ohern on r917u29n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_324(8172225) in pi_ohern on r917u29n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_325(8172226) in pi_ohern on r917u29n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_326(8172227) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_327(8172228) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_328(8172229) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_329(8172230) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_330(8172231) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_331(8172232) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_332(8172233) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_333(8172234) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_334(8172235) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_335(8172236) in pi_ohern on r917u29n02 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_336(8172237) in pi_ohern on r917u29n03 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_337(8172238) in pi_ohern on r917u29n03 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_338(8172239) in pi_ohern on r917u29n03 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_339(8172240) in pi_ohern on r917u29n03 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_340(8172241) in pi_ohern on r815u03n11 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_341(8172242) in pi_ohern on r815u03n11 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_342(8172243) in pi_ohern on r815u03n11 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_343(8172244) in pi_ohern on r815u03n12 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_344(8172245) in pi_ohern on r815u03n12 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_345(8172246) in pi_ohern on r815u03n12 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_346(8172247) in pi_ohern on r815u03n12 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_347(8172248) in pi_ohern on r815u03n12 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_348(8172249) in pi_ohern on r815u03n12 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_349(8172250) in pi_ohern on r815u03n12 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_350(8172251) in pi_ohern on r807u24n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_351(8172252) in pi_ohern on r807u24n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_352(8172253) in pi_ohern on r807u24n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_353(8172254) in pi_ohern on r807u24n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_354(8172255) in pi_ohern on r807u24n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_355(8172256) in pi_ohern on r807u24n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_356(8172257) in pi_ohern on r807u24n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_357(8172258) in pi_ohern on r807u24n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_358(8172259) in pi_ohern on r911u32n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_359(8172260) in pi_ohern on r911u32n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_360(8172261) in pi_ohern on r911u32n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_361(8172262) in pi_ohern on r913u32n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_362(8172263) in pi_ohern on r903u12n01 Nov 05 11:10:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_363(8172264) in pi_ohern on r903u26n01 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172166 in week on r816u35n03 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_907(8172265) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_908(8172266) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_909(8172267) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_910(8172268) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_911(8172269) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_912(8172270) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_913(8172271) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_914(8172272) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_915(8172273) in day on r912u32n02 Nov 05 11:11:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8170528_916(8172274) in day on r912u32n02 Nov 05 11:11:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4000(8172275) in gpu on r903u34n01 Nov 05 11:11:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4001(8172276) in gpu on r808u14n01 Nov 05 11:11:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4002(8172277) in gpu on r808u14n01 Nov 05 11:11:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7651306_3(7651391) in gpu on r4513u29n01 Nov 05 11:11:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_668(7571633) WEXITSTATUS 0 Nov 05 11:11:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_668(7571633) done Nov 05 11:11:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_332(8170484) WEXITSTATUS 0 Nov 05 11:11:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_332(8170484) done Nov 05 11:11:12 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 11:11:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3240(8132858) Nov 05 11:11:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3242(8132860) Nov 05 11:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_390(8170680) WEXITSTATUS 0 Nov 05 11:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_390(8170680) done Nov 05 11:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_266(8170403) WEXITSTATUS 0 Nov 05 11:11:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_266(8170403) done Nov 05 11:11:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8032(8172278) in scavenge on r802u09n01 Nov 05 11:11:14 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8033(8172279) in scavenge on r806u37n01 Nov 05 11:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7260(8165926) WEXITSTATUS 1 Nov 05 11:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7260(8165926) done Nov 05 11:11:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_333(8170485) WEXITSTATUS 0 Nov 05 11:11:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_333(8170485) done Nov 05 11:11:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7910(8170209) WEXITSTATUS 1 Nov 05 11:11:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7910(8170209) done Nov 05 11:11:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_383(8170673) WEXITSTATUS 0 Nov 05 11:11:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_383(8170673) done Nov 05 11:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171516 WEXITSTATUS 0 Nov 05 11:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171516 done Nov 05 11:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_433(8170839) WEXITSTATUS 0 Nov 05 11:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_433(8170839) done Nov 05 11:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_388(8170678) WEXITSTATUS 0 Nov 05 11:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_388(8170678) done Nov 05 11:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_470(8170991) WEXITSTATUS 0 Nov 05 11:11:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_470(8170991) done Nov 05 11:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_406(8170812) WEXITSTATUS 0 Nov 05 11:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_406(8170812) done Nov 05 11:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_268(8170405) WEXITSTATUS 0 Nov 05 11:11:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_268(8170405) done Nov 05 11:11:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2940(8172280) in scavenge on r806u37n01 Nov 05 11:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169921 WEXITSTATUS 0 Nov 05 11:11:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169921 done Nov 05 11:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_277(8170414) WEXITSTATUS 0 Nov 05 11:11:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_277(8170414) done Nov 05 11:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_405(8170811) WEXITSTATUS 0 Nov 05 11:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_405(8170811) done Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:11:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_373(8170663) WEXITSTATUS 0 Nov 05 11:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_373(8170663) done Nov 05 11:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_323(8170475) WEXITSTATUS 0 Nov 05 11:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_323(8170475) done Nov 05 11:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_392(8170682) WEXITSTATUS 0 Nov 05 11:11:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_392(8170682) done Nov 05 11:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7510(8166502) WEXITSTATUS 1 Nov 05 11:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7510(8166502) done Nov 05 11:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_408(8170814) WEXITSTATUS 0 Nov 05 11:11:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_408(8170814) done Nov 05 11:11:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_429(8170835) WEXITSTATUS 0 Nov 05 11:11:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_429(8170835) done Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_917(8172281) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_918(8172282) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_919(8172283) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_920(8172284) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_921(8172285) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_922(8172286) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_923(8172287) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_924(8172288) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_925(8172289) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_926(8172290) NodeList=r914u01n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_927(8172291) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_928(8172292) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_929(8172293) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_930(8172294) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_931(8172295) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_932(8172296) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_933(8172297) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_934(8172298) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_935(8172299) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_936(8172300) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_937(8172301) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_938(8172302) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_939(8172303) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_940(8172304) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_941(8172305) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_942(8172306) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_943(8172307) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_944(8172308) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_945(8172309) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_946(8172310) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_947(8172311) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_948(8172312) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_949(8172313) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_950(8172314) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_951(8172315) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_952(8172316) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_953(8172317) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_954(8172318) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_955(8172319) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_956(8172320) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_957(8172321) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_958(8172322) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_959(8172323) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_960(8172324) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_961(8172325) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_962(8172326) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_963(8172327) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_964(8172328) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_965(8172329) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_966(8172330) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_967(8172331) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_968(8172332) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_969(8172333) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_970(8172334) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_971(8172335) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_972(8172336) NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_973(8172337) NodeList=r909u29n04 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_974(8172338) NodeList=r909u29n04 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_975(8172339) NodeList=r909u29n04 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_976(8172340) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_977(8172341) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_978(8172342) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_979(8172343) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_980(8172344) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_981(8172345) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_982(8172346) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_983(8172347) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_984(8172348) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_985(8172349) NodeList=r815u35n02 #CPUs=1 Partition=day Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172350 InitPrio=48211 usec=3776 Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_740(7571705) WEXITSTATUS 0 Nov 05 11:11:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_740(7571705) done Nov 05 11:11:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_322(8170474) WEXITSTATUS 0 Nov 05 11:11:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_322(8170474) done Nov 05 11:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_434(8170840) WEXITSTATUS 0 Nov 05 11:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_434(8170840) done Nov 05 11:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_668(8168792) WEXITSTATUS 0 Nov 05 11:11:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_668(8168792) done Nov 05 11:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_608(8114875) WEXITSTATUS 0 Nov 05 11:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_608(8114875) done Nov 05 11:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_421(8170827) WEXITSTATUS 0 Nov 05 11:11:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_421(8170827) done Nov 05 11:11:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_426(8170832) WEXITSTATUS 0 Nov 05 11:11:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_426(8170832) done Nov 05 11:12:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172149 WEXITSTATUS 0 Nov 05 11:12:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172149 done Nov 05 11:12:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4000(8172275) WEXITSTATUS 0 Nov 05 11:12:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4000(8172275) done Nov 05 11:12:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_384(8170674) WEXITSTATUS 0 Nov 05 11:12:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_384(8170674) done Nov 05 11:12:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172351 InitPrio=48210 usec=3110 Nov 05 11:12:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7803(8169346) WEXITSTATUS 1 Nov 05 11:12:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7803(8169346) done Nov 05 11:12:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172351 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 11:12:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4001(8172276) WEXITSTATUS 0 Nov 05 11:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4001(8172276) done Nov 05 11:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_330(8170482) WEXITSTATUS 0 Nov 05 11:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_330(8170482) done Nov 05 11:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8022(8171989) WEXITSTATUS 1 Nov 05 11:12:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8022(8171989) done Nov 05 11:12:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_401(8170807) WEXITSTATUS 0 Nov 05 11:12:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_401(8170807) done Nov 05 11:12:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172352 InitPrio=34224 usec=4164 Nov 05 11:12:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_329(8170481) WEXITSTATUS 0 Nov 05 11:12:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_329(8170481) done Nov 05 11:12:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_471(8170992) WEXITSTATUS 0 Nov 05 11:12:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_471(8170992) done Nov 05 11:12:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7323(8166014) WEXITSTATUS 1 Nov 05 11:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7323(8166014) done Nov 05 11:12:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6592(8164634) WEXITSTATUS 1 Nov 05 11:12:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6592(8164634) done Nov 05 11:12:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_466(8170987) WEXITSTATUS 0 Nov 05 11:12:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_466(8170987) done Nov 05 11:12:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_361(8170533) WEXITSTATUS 0 Nov 05 11:12:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_361(8170533) done Nov 05 11:12:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_409(8170815) WEXITSTATUS 0 Nov 05 11:12:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_409(8170815) done Nov 05 11:12:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:44 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5322(8158850) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:12:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5322(8158850) Nov 05 11:12:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172350 NodeList=r806u37n01 #CPUs=16 Partition=gpu Nov 05 11:12:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_412(8170818) WEXITSTATUS 0 Nov 05 11:12:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_412(8170818) done Nov 05 11:12:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172352 NodeList=r904u07n04 #CPUs=4 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_986(8172353) NodeList=r904u07n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_987(8172354) NodeList=r904u07n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_988(8172355) NodeList=r904u07n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_989(8172356) NodeList=r904u07n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_990(8172357) NodeList=r904u07n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_991(8172358) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_992(8172359) NodeList=r805u06n02 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_993(8172360) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_994(8172361) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_995(8172362) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_996(8172363) NodeList=r805u14n04 #CPUs=1 Partition=day Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_759(7571724) WEXITSTATUS 0 Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_759(7571724) done Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_469(8170990) WEXITSTATUS 0 Nov 05 11:12:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_469(8170990) done Nov 05 11:12:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_331(8170483) WEXITSTATUS 0 Nov 05 11:12:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_331(8170483) done Nov 05 11:12:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_403(8170809) WEXITSTATUS 0 Nov 05 11:12:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_403(8170809) done Nov 05 11:12:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172364 InitPrio=17501 usec=7185 Nov 05 11:12:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:12:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_416(8170822) WEXITSTATUS 0 Nov 05 11:12:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_416(8170822) done Nov 05 11:13:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172350 WEXITSTATUS 1 Nov 05 11:13:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172350 done Nov 05 11:13:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7456(8166292) WEXITSTATUS 1 Nov 05 11:13:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7456(8166292) done Nov 05 11:13:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_410(8170816) WEXITSTATUS 0 Nov 05 11:13:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_410(8170816) done Nov 05 11:13:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7976(8171124) WEXITSTATUS 1 Nov 05 11:13:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7976(8171124) done Nov 05 11:13:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_420(8170826) WEXITSTATUS 0 Nov 05 11:13:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_420(8170826) done Nov 05 11:13:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_370(8170660) WEXITSTATUS 0 Nov 05 11:13:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_370(8170660) done Nov 05 11:13:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_480(8171001) WEXITSTATUS 0 Nov 05 11:13:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_480(8171001) done Nov 05 11:13:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172150 WEXITSTATUS 0 Nov 05 11:13:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172150 done Nov 05 11:13:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7949(8170650) WEXITSTATUS 1 Nov 05 11:13:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7949(8170650) done Nov 05 11:13:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6695(8164797) WEXITSTATUS 1 Nov 05 11:13:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6695(8164797) done Nov 05 11:13:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:13:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:13:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_402(8170808) WEXITSTATUS 0 Nov 05 11:13:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_402(8170808) done Nov 05 11:13:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_432(8170838) WEXITSTATUS 0 Nov 05 11:13:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_432(8170838) done Nov 05 11:13:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_372(8170662) WEXITSTATUS 0 Nov 05 11:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_372(8170662) done Nov 05 11:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7328(8166025) WEXITSTATUS 1 Nov 05 11:13:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7328(8166025) done Nov 05 11:13:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:13:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_997(8172365) NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 11:13:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_998(8172366) NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 11:13:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170528_999(8170528) NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 11:13:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172364 NodeList=r912u09n01 #CPUs=1 Partition=bigmem Nov 05 11:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_394(8170800) WEXITSTATUS 0 Nov 05 11:14:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_394(8170800) done Nov 05 11:14:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:14:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172167 uid 25787 Nov 05 11:14:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_492(8171013) WEXITSTATUS 0 Nov 05 11:14:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_492(8171013) done Nov 05 11:14:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:14:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_425(8170831) WEXITSTATUS 0 Nov 05 11:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_425(8170831) done Nov 05 11:14:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_435(8170841) WEXITSTATUS 0 Nov 05 11:14:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_435(8170841) done Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_364(8172367) in pi_ohern on r801u32n02 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_365(8172368) in pi_ohern on r917u13n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_366(8172369) in pi_ohern on r917u13n02 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_367(8172370) in pi_ohern on r917u13n02 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_368(8172371) in pi_ohern on r917u13n04 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_369(8172372) in pi_ohern on r917u13n04 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_370(8172373) in pi_ohern on r917u13n04 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_371(8172374) in pi_ohern on r917u13n04 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_372(8172375) in pi_ohern on r917u29n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_373(8172376) in pi_ohern on r917u29n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_374(8172377) in pi_ohern on r917u29n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_375(8172378) in pi_ohern on r917u29n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_376(8172379) in pi_ohern on r917u29n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_377(8172380) in pi_ohern on r917u29n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_378(8172381) in pi_ohern on r917u29n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_379(8172382) in pi_ohern on r917u29n02 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_380(8172383) in pi_ohern on r917u29n02 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_381(8172384) in pi_ohern on r917u29n03 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_382(8172385) in pi_ohern on r917u29n04 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_383(8172386) in pi_ohern on r917u29n04 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_384(8172387) in pi_ohern on r815u03n11 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_385(8172388) in pi_ohern on r815u03n11 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_386(8172389) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_387(8172390) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_388(8172391) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_389(8172392) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_390(8172393) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_391(8172394) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_392(8172395) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_393(8172396) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_394(8172397) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_395(8172398) in pi_ohern on r815u03n12 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_396(8172399) in pi_ohern on r807u24n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_397(8172400) in pi_ohern on r911u32n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_398(8172401) in pi_ohern on r911u32n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_399(8172402) in pi_ohern on r911u32n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_400(8172403) in pi_ohern on r903u12n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_401(8172404) in pi_ohern on r903u24n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_402(8172405) in pi_ohern on r903u24n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_403(8172406) in pi_ohern on r903u24n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_404(8172407) in pi_ohern on r903u24n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_405(8172408) in pi_ohern on r903u24n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_406(8172409) in pi_ohern on r903u24n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_407(8172410) in pi_ohern on r913u32n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_408(8172411) in pi_ohern on r913u32n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_409(8172412) in pi_ohern on r913u35n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4003(8172413) in gpu on r808u14n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4004(8172414) in gpu on r903u34n01 Nov 05 11:14:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r805u09n02 Nov 05 11:14:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_486(8171007) WEXITSTATUS 0 Nov 05 11:14:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_486(8171007) done Nov 05 11:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_399(8170805) WEXITSTATUS 0 Nov 05 11:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_399(8170805) done Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8034(8172415) in scavenge on r801u04n01 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8035(8172416) in scavenge on r802u09n03 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8036(8172417) in scavenge on r903u01n01 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8037(8172418) in scavenge on r915u35n02 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8038(8172419) in scavenge on r816u29n11 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8039(8172420) in scavenge on r815u09n10 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8040(8172421) in scavenge on r815u09n12 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8041(8172422) in scavenge on r816u15n10 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8042(8172423) in scavenge on r816u15n12 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8043(8172424) in scavenge on r918u05n03 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8044(8172425) in scavenge on r909u10n03 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8045(8172426) in scavenge on r909u10n03 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8046(8172427) in scavenge on r909u10n03 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8047(8172428) in scavenge on r909u13n04 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8048(8172429) in scavenge on r806u37n01 Nov 05 11:14:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8049(8172430) in scavenge on r806u37n01 Nov 05 11:14:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_313(8170465) WEXITSTATUS 0 Nov 05 11:14:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_313(8170465) done Nov 05 11:14:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2941(8172431) in scavenge on r815u09n10 Nov 05 11:14:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_490(8171011) WEXITSTATUS 0 Nov 05 11:14:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_490(8171011) done Nov 05 11:14:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_481(8171002) WEXITSTATUS 0 Nov 05 11:14:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_481(8171002) done Nov 05 11:14:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_448(8170969) WEXITSTATUS 0 Nov 05 11:14:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_448(8170969) done Nov 05 11:14:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_309(8170461) WEXITSTATUS 0 Nov 05 11:14:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_309(8170461) done Nov 05 11:14:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_66(8135765) WEXITSTATUS 0 Nov 05 11:14:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_66(8135765) done Nov 05 11:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_395(8170801) WEXITSTATUS 0 Nov 05 11:14:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_395(8170801) done Nov 05 11:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172151 WEXITSTATUS 0 Nov 05 11:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172151 done Nov 05 11:14:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_452(8170973) WEXITSTATUS 0 Nov 05 11:14:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_452(8170973) done Nov 05 11:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_396(8170802) WEXITSTATUS 0 Nov 05 11:14:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_396(8170802) done Nov 05 11:14:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_2000(8125740) Nov 05 11:14:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_485(8171006) WEXITSTATUS 0 Nov 05 11:14:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_485(8171006) done Nov 05 11:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_440(8170961) WEXITSTATUS 0 Nov 05 11:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_440(8170961) done Nov 05 11:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_436(8170842) WEXITSTATUS 0 Nov 05 11:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_436(8170842) done Nov 05 11:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_478(8170999) WEXITSTATUS 0 Nov 05 11:14:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_478(8170999) done Nov 05 11:14:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_324(8170476) WEXITSTATUS 0 Nov 05 11:14:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_324(8170476) done Nov 05 11:14:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_397(8170803) WEXITSTATUS 0 Nov 05 11:14:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_397(8170803) done Nov 05 11:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_482(8171003) WEXITSTATUS 0 Nov 05 11:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_482(8171003) done Nov 05 11:15:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8172432 NodeList=(null) usec=27231 Nov 05 11:15:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172432 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 11:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_419(8170825) WEXITSTATUS 0 Nov 05 11:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_419(8170825) done Nov 05 11:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6732(8164847) WEXITSTATUS 1 Nov 05 11:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6732(8164847) done Nov 05 11:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_450(8170971) WEXITSTATUS 0 Nov 05 11:15:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_450(8170971) done Nov 05 11:15:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172433 InitPrio=16342 usec=7322 Nov 05 11:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_544(8171180) WEXITSTATUS 0 Nov 05 11:15:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_544(8171180) done Nov 05 11:15:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:15:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_404(8170810) WEXITSTATUS 0 Nov 05 11:15:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_404(8170810) done Nov 05 11:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168455 WEXITSTATUS 0 Nov 05 11:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168455 done Nov 05 11:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_375(8170665) WEXITSTATUS 0 Nov 05 11:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_375(8170665) done Nov 05 11:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_441(8170962) WEXITSTATUS 0 Nov 05 11:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_441(8170962) done Nov 05 11:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7538(8166670) WEXITSTATUS 1 Nov 05 11:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7538(8166670) done Nov 05 11:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_371(8170661) WEXITSTATUS 0 Nov 05 11:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_371(8170661) done Nov 05 11:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_428(8170834) WEXITSTATUS 0 Nov 05 11:15:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_428(8170834) done Nov 05 11:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_493(8171014) WEXITSTATUS 0 Nov 05 11:15:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_493(8171014) done Nov 05 11:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3725(8163692) WEXITSTATUS 0 Nov 05 11:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3725(8163692) done Nov 05 11:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_431(8170837) WEXITSTATUS 0 Nov 05 11:15:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_431(8170837) done Nov 05 11:15:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_483(8171004) WEXITSTATUS 0 Nov 05 11:15:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_483(8171004) done Nov 05 11:15:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172434 InitPrio=16342 usec=3946 Nov 05 11:15:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172435 InitPrio=27319 usec=4039 Nov 05 11:15:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_387(8170677) WEXITSTATUS 0 Nov 05 11:15:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_387(8170677) done Nov 05 11:15:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_424(8170830) WEXITSTATUS 0 Nov 05 11:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_424(8170830) done Nov 05 11:15:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:15:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167599 WEXITSTATUS 0 Nov 05 11:15:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167599 done Nov 05 11:15:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_453(8170974) WEXITSTATUS 0 Nov 05 11:15:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_453(8170974) done Nov 05 11:15:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172433 uid 20225 Nov 05 11:15:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172434 uid 20225 Nov 05 11:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_487(8171008) WEXITSTATUS 0 Nov 05 11:15:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_487(8171008) done Nov 05 11:15:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_427(8170833) WEXITSTATUS 0 Nov 05 11:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_427(8170833) done Nov 05 11:15:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172435 NodeList=r805u09n02 #CPUs=4 Partition=day Nov 05 11:15:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8168321 NodeList=r816u15n08 #CPUs=8 Partition=day Nov 05 11:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_484(8171005) WEXITSTATUS 0 Nov 05 11:15:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_484(8171005) done Nov 05 11:15:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:15:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_538(8171174) WEXITSTATUS 0 Nov 05 11:15:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_538(8171174) done Nov 05 11:16:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6416(8164189) WEXITSTATUS 1 Nov 05 11:16:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6416(8164189) done Nov 05 11:16:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_437(8170843) WEXITSTATUS 0 Nov 05 11:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_437(8170843) done Nov 05 11:16:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7893(8170100) WEXITSTATUS 1 Nov 05 11:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7893(8170100) done Nov 05 11:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_423(8170829) WEXITSTATUS 0 Nov 05 11:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_423(8170829) done Nov 05 11:16:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_475(8170996) WEXITSTATUS 0 Nov 05 11:16:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_475(8170996) done Nov 05 11:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_462(8170983) WEXITSTATUS 0 Nov 05 11:16:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_462(8170983) done Nov 05 11:16:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_439(8170960) WEXITSTATUS 0 Nov 05 11:16:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_439(8170960) done Nov 05 11:16:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_629(8171380) WEXITSTATUS 0 Nov 05 11:16:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_629(8171380) done Nov 05 11:16:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_455(8170976) WEXITSTATUS 0 Nov 05 11:16:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_455(8170976) done Nov 05 11:16:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7667(8168303) WEXITSTATUS 1 Nov 05 11:16:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7667(8168303) done Nov 05 11:16:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:32 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:33 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:36 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:36 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:37 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6467(8164372) WEXITSTATUS 1 Nov 05 11:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6467(8164372) done Nov 05 11:16:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172435 uid 25787 Nov 05 11:16:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_474(8170995) WEXITSTATUS 0 Nov 05 11:16:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_474(8170995) done Nov 05 11:16:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7944(8170645) WEXITSTATUS 1 Nov 05 11:16:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7944(8170645) done Nov 05 11:16:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_54(8135753) WEXITSTATUS 0 Nov 05 11:16:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_54(8135753) done Nov 05 11:16:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172436 InitPrio=16342 usec=4367 Nov 05 11:16:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_467(8170988) WEXITSTATUS 0 Nov 05 11:16:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_467(8170988) done Nov 05 11:16:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_311(7572479) WEXITSTATUS 0 Nov 05 11:16:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_311(7572479) done Nov 05 11:16:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_432(7572600) WEXITSTATUS 0 Nov 05 11:16:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_432(7572600) done Nov 05 11:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_454(8170975) WEXITSTATUS 0 Nov 05 11:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_454(8170975) done Nov 05 11:16:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_495(8171016) WEXITSTATUS 0 Nov 05 11:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_495(8171016) done Nov 05 11:16:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_477(8170998) WEXITSTATUS 0 Nov 05 11:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_477(8170998) done Nov 05 11:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_443(8170964) WEXITSTATUS 0 Nov 05 11:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_443(8170964) done Nov 05 11:17:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_465(8170986) WEXITSTATUS 0 Nov 05 11:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_465(8170986) done Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_410(8172437) in pi_ohern on r801u32n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_411(8172438) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_412(8172439) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_413(8172440) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_414(8172441) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_415(8172442) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_416(8172443) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_417(8172444) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_418(8172445) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_419(8172446) in pi_ohern on r917u13n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_420(8172447) in pi_ohern on r917u13n03 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_421(8172448) in pi_ohern on r917u13n03 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_422(8172449) in pi_ohern on r917u13n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_423(8172450) in pi_ohern on r917u13n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_424(8172451) in pi_ohern on r917u13n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_425(8172452) in pi_ohern on r917u13n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_426(8172453) in pi_ohern on r917u13n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_427(8172454) in pi_ohern on r917u13n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_428(8172455) in pi_ohern on r917u29n02 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_429(8172456) in pi_ohern on r917u29n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_430(8172457) in pi_ohern on r917u29n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_431(8172458) in pi_ohern on r917u29n04 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_432(8172459) in pi_ohern on r815u03n12 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_433(8172460) in pi_ohern on r815u03n12 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_434(8172461) in pi_ohern on r815u03n12 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_435(8172462) in pi_ohern on r807u22n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_436(8172463) in pi_ohern on r807u22n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_437(8172464) in pi_ohern on r807u22n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_438(8172465) in pi_ohern on r807u22n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_439(8172466) in pi_ohern on r911u32n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_440(8172467) in pi_ohern on r911u32n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_441(8172468) in pi_ohern on r911u32n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_442(8172469) in pi_ohern on r911u32n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_443(8172470) in pi_ohern on r913u32n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_444(8172471) in pi_ohern on r913u35n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_445(8172472) in pi_ohern on r913u35n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_446(8172473) in pi_ohern on r913u35n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_447(8172474) in pi_ohern on r913u35n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_448(8172475) in pi_ohern on r913u35n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_449(8172476) in pi_ohern on r913u35n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_450(8172477) in pi_ohern on r903u12n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_451(8172478) in pi_ohern on r903u12n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_452(8172479) in pi_ohern on r903u12n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_453(8172480) in pi_ohern on r903u12n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_454(8172481) in pi_ohern on r903u24n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_455(8172482) in pi_ohern on r903u24n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_456(8172483) in pi_ohern on r903u26n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_457(8172484) in pi_ohern on r903u26n01 Nov 05 11:17:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_458(8172485) in pi_ohern on r903u26n01 Nov 05 11:17:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4005(8172486) in gpu on r4513u26n01 Nov 05 11:17:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_451(8170972) WEXITSTATUS 0 Nov 05 11:17:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_451(8170972) done Nov 05 11:17:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172487 InitPrio=27319 usec=4121 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8050(8172488) in scavenge on r907u01n04 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8051(8172489) in scavenge on r903u07n01 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8052(8172490) in scavenge on r815u03n10 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8053(8172491) in scavenge on r815u23n08 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8054(8172492) in scavenge on r816u03n09 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8055(8172493) in scavenge on r806u14n02 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8056(8172494) in scavenge on r909u07n04 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8057(8172495) in scavenge on r909u13n04 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8058(8172496) in scavenge on r912u32n03 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8059(8172497) in scavenge on r913u05n01 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8060(8172498) in scavenge on r815u15n02 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8061(8172499) in scavenge on r815u15n02 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8062(8172500) in scavenge on r815u15n02 Nov 05 11:17:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8063(8172501) in scavenge on r815u15n02 Nov 05 11:17:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 11:17:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_489(8171010) WEXITSTATUS 0 Nov 05 11:17:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_489(8171010) done Nov 05 11:17:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_442(8170963) WEXITSTATUS 0 Nov 05 11:17:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_442(8170963) done Nov 05 11:17:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2942(8172502) in scavenge on r815u03n10 Nov 05 11:17:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2943(8172503) in scavenge on r815u03n10 Nov 05 11:17:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2944(8172504) in scavenge on r815u03n10 Nov 05 11:17:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2945(8172505) in scavenge on r815u03n10 Nov 05 11:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3991(8171506) WEXITSTATUS 0 Nov 05 11:17:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3991(8171506) done Nov 05 11:17:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_644(8171395) WEXITSTATUS 0 Nov 05 11:17:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_644(8171395) done Nov 05 11:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_468(8170989) WEXITSTATUS 0 Nov 05 11:17:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_468(8170989) done Nov 05 11:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8052(8172490) WEXITSTATUS 1 Nov 05 11:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8052(8172490) done Nov 05 11:17:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167603 WEXITSTATUS 0 Nov 05 11:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167603 done Nov 05 11:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_460(8170981) WEXITSTATUS 0 Nov 05 11:17:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_460(8170981) done Nov 05 11:17:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6899(8165146) WEXITSTATUS 1 Nov 05 11:17:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6899(8165146) done Nov 05 11:17:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:17:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7845(8169562) WEXITSTATUS 1 Nov 05 11:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7845(8169562) done Nov 05 11:17:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_491(8171012) WEXITSTATUS 0 Nov 05 11:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_491(8171012) done Nov 05 11:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_547(8171183) WEXITSTATUS 0 Nov 05 11:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_547(8171183) done Nov 05 11:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_458(8170979) WEXITSTATUS 0 Nov 05 11:17:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_458(8170979) done Nov 05 11:17:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172506 InitPrio=29089 usec=13675 Nov 05 11:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_456(8170977) WEXITSTATUS 0 Nov 05 11:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_456(8170977) done Nov 05 11:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_555(8171191) WEXITSTATUS 0 Nov 05 11:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_555(8171191) done Nov 05 11:17:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:17:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172506 NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 11:17:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172506 WEXITSTATUS 0 Nov 05 11:17:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172506 done Nov 05 11:17:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_785(7571750) WEXITSTATUS 0 Nov 05 11:17:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_785(7571750) done Nov 05 11:17:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_476(8170997) WEXITSTATUS 0 Nov 05 11:17:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_476(8170997) done Nov 05 11:17:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_645(8171396) WEXITSTATUS 0 Nov 05 11:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_645(8171396) done Nov 05 11:18:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172507 InitPrio=33564 usec=2032 Nov 05 11:18:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_552(8171188) WEXITSTATUS 0 Nov 05 11:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_552(8171188) done Nov 05 11:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_534(8171170) WEXITSTATUS 0 Nov 05 11:18:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_534(8171170) done Nov 05 11:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167605 WEXITSTATUS 0 Nov 05 11:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167605 done Nov 05 11:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_479(8171000) WEXITSTATUS 0 Nov 05 11:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_479(8171000) done Nov 05 11:18:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7575(8166813) WEXITSTATUS 1 Nov 05 11:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7575(8166813) done Nov 05 11:18:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_556(8171192) WEXITSTATUS 0 Nov 05 11:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_556(8171192) done Nov 05 11:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_494(8171015) WEXITSTATUS 0 Nov 05 11:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_494(8171015) done Nov 05 11:18:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_508(8171144) WEXITSTATUS 0 Nov 05 11:18:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_508(8171144) done Nov 05 11:18:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172508 InitPrio=16342 usec=4111 Nov 05 11:18:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7756(8168910) WEXITSTATUS 1 Nov 05 11:18:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7756(8168910) done Nov 05 11:18:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_473(8170994) WEXITSTATUS 0 Nov 05 11:18:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_473(8170994) done Nov 05 11:18:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172509 InitPrio=29089 usec=4592 Nov 05 11:18:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_369(8170659) WEXITSTATUS 0 Nov 05 11:18:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_369(8170659) done Nov 05 11:18:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_557(8171193) WEXITSTATUS 0 Nov 05 11:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_557(8171193) done Nov 05 11:18:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172508 uid 20225 Nov 05 11:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_457(8170978) WEXITSTATUS 0 Nov 05 11:18:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_457(8170978) done Nov 05 11:18:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8060(8172498) WEXITSTATUS 1 Nov 05 11:18:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8060(8172498) done Nov 05 11:18:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_641(8171392) WEXITSTATUS 0 Nov 05 11:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_641(8171392) done Nov 05 11:18:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_558(8171194) WEXITSTATUS 0 Nov 05 11:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_558(8171194) done Nov 05 11:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_535(8171171) WEXITSTATUS 0 Nov 05 11:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_535(8171171) done Nov 05 11:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_652(8171403) WEXITSTATUS 0 Nov 05 11:18:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_652(8171403) done Nov 05 11:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_400(8170806) WEXITSTATUS 0 Nov 05 11:18:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_400(8170806) done Nov 05 11:18:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172510 InitPrio=16342 usec=3917 Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172509 NodeList=r805u09n02 #CPUs=1 Partition=day Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172487 NodeList=r912u32n04 #CPUs=4 Partition=day Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172507 NodeList=r815u23n10 #CPUs=12 Partition=scavenge Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172511 InitPrio=29089 usec=3914 Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172509 WEXITSTATUS 0 Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172509 done Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_283(7572451) WEXITSTATUS 0 Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_283(7572451) done Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_946(7573115) WEXITSTATUS 0 Nov 05 11:18:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_946(7573115) done Nov 05 11:18:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172512 InitPrio=32855 usec=7378 Nov 05 11:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4004(8172414) WEXITSTATUS 0 Nov 05 11:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4004(8172414) done Nov 05 11:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_463(8170984) WEXITSTATUS 0 Nov 05 11:18:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_463(8170984) done Nov 05 11:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_447(8170968) WEXITSTATUS 0 Nov 05 11:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_447(8170968) done Nov 05 11:18:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172512 NodeList=r806u23n04 #CPUs=2 Partition=devel Nov 05 11:18:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167602 WEXITSTATUS 0 Nov 05 11:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167602 done Nov 05 11:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_619(8171370) WEXITSTATUS 0 Nov 05 11:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_619(8171370) done Nov 05 11:19:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_536(8171172) WEXITSTATUS 0 Nov 05 11:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_536(8171172) done Nov 05 11:19:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_546(8171182) WEXITSTATUS 0 Nov 05 11:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_546(8171182) done Nov 05 11:19:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_545(8171181) WEXITSTATUS 0 Nov 05 11:19:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_545(8171181) done Nov 05 11:19:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_543(8171179) WEXITSTATUS 0 Nov 05 11:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_543(8171179) done Nov 05 11:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_449(8170970) WEXITSTATUS 0 Nov 05 11:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_449(8170970) done Nov 05 11:19:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_628(8171379) WEXITSTATUS 0 Nov 05 11:19:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_628(8171379) done Nov 05 11:19:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172513 InitPrio=56564 usec=3244 Nov 05 11:19:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172513 NodeList=r806u25n01 #CPUs=1 Partition=devel Nov 05 11:19:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_541(8171177) WEXITSTATUS 0 Nov 05 11:19:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_541(8171177) done Nov 05 11:19:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_497(8171018) WEXITSTATUS 0 Nov 05 11:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_497(8171018) done Nov 05 11:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_554(8171190) WEXITSTATUS 0 Nov 05 11:19:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_554(8171190) done Nov 05 11:19:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_618(8171369) WEXITSTATUS 0 Nov 05 11:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_618(8171369) done Nov 05 11:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_639(8171390) WEXITSTATUS 0 Nov 05 11:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_639(8171390) done Nov 05 11:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_459(8170980) WEXITSTATUS 0 Nov 05 11:19:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_459(8170980) done Nov 05 11:19:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_445(8170966) WEXITSTATUS 0 Nov 05 11:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_445(8170966) done Nov 05 11:19:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_579(8171330) WEXITSTATUS 0 Nov 05 11:19:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_579(8171330) done Nov 05 11:19:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_540(8171176) WEXITSTATUS 0 Nov 05 11:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_540(8171176) done Nov 05 11:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_515(8171151) WEXITSTATUS 0 Nov 05 11:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_515(8171151) done Nov 05 11:19:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_461(8170982) WEXITSTATUS 0 Nov 05 11:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_461(8170982) done Nov 05 11:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_464(8170985) WEXITSTATUS 0 Nov 05 11:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_464(8170985) done Nov 05 11:19:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172514 InitPrio=20039 usec=4030 Nov 05 11:19:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170956 WEXITSTATUS 0 Nov 05 11:19:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170956 done Nov 05 11:19:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7809(8169352) WEXITSTATUS 1 Nov 05 11:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7809(8169352) done Nov 05 11:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_502(8171138) WEXITSTATUS 0 Nov 05 11:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_502(8171138) done Nov 05 11:19:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4002(8172277) WEXITSTATUS 0 Nov 05 11:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4002(8172277) done Nov 05 11:19:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_510(8171146) WEXITSTATUS 0 Nov 05 11:19:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_510(8171146) done Nov 05 11:19:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7666(8168302) WEXITSTATUS 1 Nov 05 11:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7666(8168302) done Nov 05 11:19:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172511 NodeList=r815u23n05 #CPUs=16 Partition=day Nov 05 11:19:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_649(8171400) WEXITSTATUS 0 Nov 05 11:19:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_649(8171400) done Nov 05 11:19:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_650(8171401) WEXITSTATUS 0 Nov 05 11:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_650(8171401) done Nov 05 11:19:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_522(8171158) WEXITSTATUS 0 Nov 05 11:19:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_522(8171158) done Nov 05 11:19:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_459(8172515) in pi_ohern on r917u13n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_460(8172516) in pi_ohern on r917u13n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_461(8172517) in pi_ohern on r917u13n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_462(8172518) in pi_ohern on r917u13n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_463(8172519) in pi_ohern on r917u13n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_464(8172520) in pi_ohern on r917u13n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_465(8172521) in pi_ohern on r917u13n03 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_466(8172522) in pi_ohern on r917u13n03 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_467(8172523) in pi_ohern on r917u13n03 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_468(8172524) in pi_ohern on r917u13n04 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_469(8172525) in pi_ohern on r917u13n04 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_470(8172526) in pi_ohern on r917u13n04 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_471(8172527) in pi_ohern on r917u13n04 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_472(8172528) in pi_ohern on r917u29n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_473(8172529) in pi_ohern on r917u29n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_474(8172530) in pi_ohern on r917u29n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_475(8172531) in pi_ohern on r917u29n02 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_476(8172532) in pi_ohern on r917u29n03 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_477(8172533) in pi_ohern on r917u29n04 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_478(8172534) in pi_ohern on r917u29n04 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_479(8172535) in pi_ohern on r917u29n04 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_480(8172536) in pi_ohern on r917u29n04 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_481(8172537) in pi_ohern on r815u03n11 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_482(8172538) in pi_ohern on r815u03n11 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_483(8172539) in pi_ohern on r815u03n11 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_484(8172540) in pi_ohern on r815u03n11 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_485(8172541) in pi_ohern on r815u03n12 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_486(8172542) in pi_ohern on r815u03n12 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_487(8172543) in pi_ohern on r911u32n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_488(8172544) in pi_ohern on r911u32n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_489(8172545) in pi_ohern on r807u22n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_490(8172546) in pi_ohern on r807u22n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_491(8172547) in pi_ohern on r807u24n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_492(8172548) in pi_ohern on r807u24n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_493(8172549) in pi_ohern on r913u32n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_494(8172550) in pi_ohern on r913u32n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_495(8172551) in pi_ohern on r913u35n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_496(8172552) in pi_ohern on r913u35n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_497(8172553) in pi_ohern on r913u35n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_498(8172554) in pi_ohern on r913u35n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_499(8172555) in pi_ohern on r913u35n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_500(8172556) in pi_ohern on r903u12n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_501(8172557) in pi_ohern on r903u12n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_502(8172558) in pi_ohern on r903u24n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_503(8172559) in pi_ohern on r903u24n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_504(8172560) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_505(8172561) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_506(8172562) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_507(8172563) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_508(8172564) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_509(8172565) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_510(8172566) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_511(8172567) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_512(8172568) in pi_ohern on r903u26n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4006(8172569) in gpu on r808u14n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4007(8172570) in gpu on r903u34n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4008(8172571) in gpu on r903u34n01 Nov 05 11:19:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r805u09n02 Nov 05 11:20:00 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:116 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 11:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167604 WEXITSTATUS 0 Nov 05 11:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167604 done Nov 05 11:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_446(8170967) WEXITSTATUS 0 Nov 05 11:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_446(8170967) done Nov 05 11:20:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_637(8171388) WEXITSTATUS 0 Nov 05 11:20:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_637(8171388) done Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8064(8172572) in scavenge on r802u30n03 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8065(8172573) in scavenge on r815u03n10 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8066(8172574) in scavenge on r815u29n11 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8067(8172575) in scavenge on r816u09n09 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8068(8172576) in scavenge on r816u23n09 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8069(8172577) in scavenge on r816u29n09 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8070(8172578) in scavenge on r807u03n01 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8071(8172579) in scavenge on r918u05n04 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8072(8172580) in scavenge on r913u09n02 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8073(8172581) in scavenge on r913u13n03 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8074(8172582) in scavenge on r913u13n03 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8075(8172583) in scavenge on r815u15n02 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8076(8172584) in scavenge on r815u23n05 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8077(8172585) in scavenge on r816u15n04 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8078(8172586) in scavenge on r816u15n04 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8079(8172587) in scavenge on r816u15n04 Nov 05 11:20:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8080(8172588) in scavenge on r816u15n04 Nov 05 11:20:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4763(8157334) WEXITSTATUS 1 Nov 05 11:20:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4763(8157334) done Nov 05 11:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_636(8171387) WEXITSTATUS 0 Nov 05 11:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_636(8171387) done Nov 05 11:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_614(8171365) WEXITSTATUS 0 Nov 05 11:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_614(8171365) done Nov 05 11:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_501(8171137) WEXITSTATUS 0 Nov 05 11:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_501(8171137) done Nov 05 11:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_647(8171398) WEXITSTATUS 0 Nov 05 11:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_647(8171398) done Nov 05 11:20:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:900 Nov 05 11:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_513(8171149) WEXITSTATUS 0 Nov 05 11:20:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_513(8171149) done Nov 05 11:20:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2946(8172589) in scavenge on r816u23n09 Nov 05 11:20:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2947(8172590) in scavenge on r816u23n09 Nov 05 11:20:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2948(8172591) in scavenge on r816u29n09 Nov 05 11:20:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2949(8172592) in scavenge on r816u29n09 Nov 05 11:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_627(8171378) WEXITSTATUS 0 Nov 05 11:20:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_627(8171378) done Nov 05 11:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_523(8171159) WEXITSTATUS 0 Nov 05 11:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_523(8171159) done Nov 05 11:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_553(8171189) WEXITSTATUS 0 Nov 05 11:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_553(8171189) done Nov 05 11:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167600 WEXITSTATUS 0 Nov 05 11:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167600 done Nov 05 11:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_519(8171155) WEXITSTATUS 0 Nov 05 11:20:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_519(8171155) done Nov 05 11:20:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8172593 NodeList=(null) usec=38319 Nov 05 11:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172593 NodeList=r908u33n01 #CPUs=4 Partition=admintest Nov 05 11:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_600(8171351) WEXITSTATUS 0 Nov 05 11:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_600(8171351) done Nov 05 11:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_537(8171173) WEXITSTATUS 0 Nov 05 11:20:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_537(8171173) done Nov 05 11:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_527(8171163) WEXITSTATUS 0 Nov 05 11:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_527(8171163) done Nov 05 11:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_625(8171376) WEXITSTATUS 0 Nov 05 11:20:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_625(8171376) done Nov 05 11:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_509(8171145) WEXITSTATUS 0 Nov 05 11:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_509(8171145) done Nov 05 11:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_593(8171344) WEXITSTATUS 0 Nov 05 11:20:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_593(8171344) done Nov 05 11:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_620(8171371) WEXITSTATUS 0 Nov 05 11:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_620(8171371) done Nov 05 11:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7943(8170644) WEXITSTATUS 1 Nov 05 11:20:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7943(8170644) done Nov 05 11:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8041(8172422) WEXITSTATUS 1 Nov 05 11:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8041(8172422) done Nov 05 11:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_642(8171393) WEXITSTATUS 0 Nov 05 11:20:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_642(8171393) done Nov 05 11:20:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_651(8171402) WEXITSTATUS 0 Nov 05 11:20:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_651(8171402) done Nov 05 11:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_532(8171168) WEXITSTATUS 0 Nov 05 11:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_532(8171168) done Nov 05 11:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_516(8171152) WEXITSTATUS 0 Nov 05 11:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_516(8171152) done Nov 05 11:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_496(8171017) WEXITSTATUS 0 Nov 05 11:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_496(8171017) done Nov 05 11:21:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_529(8171165) WEXITSTATUS 0 Nov 05 11:21:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_529(8171165) done Nov 05 11:21:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_514(8171150) WEXITSTATUS 0 Nov 05 11:21:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_514(8171150) done Nov 05 11:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_550(8171186) WEXITSTATUS 0 Nov 05 11:21:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_550(8171186) done Nov 05 11:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_638(8171389) WEXITSTATUS 0 Nov 05 11:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_638(8171389) done Nov 05 11:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_626(8171377) WEXITSTATUS 0 Nov 05 11:21:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_626(8171377) done Nov 05 11:21:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172487 uid 25787 Nov 05 11:21:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_488(8171009) WEXITSTATUS 0 Nov 05 11:21:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_488(8171009) done Nov 05 11:21:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_438(8170959) WEXITSTATUS 0 Nov 05 11:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_438(8170959) done Nov 05 11:21:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4006(8172569) WEXITSTATUS 0 Nov 05 11:21:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4006(8172569) done Nov 05 11:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_621(8171372) WEXITSTATUS 0 Nov 05 11:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_621(8171372) done Nov 05 11:21:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_567(8171318) WEXITSTATUS 0 Nov 05 11:21:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_567(8171318) done Nov 05 11:21:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1957(8163589) WEXITSTATUS 0 Nov 05 11:21:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1957(8163589) done Nov 05 11:21:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_635(8171386) WEXITSTATUS 0 Nov 05 11:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_635(8171386) done Nov 05 11:21:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_623(8171374) WEXITSTATUS 0 Nov 05 11:21:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_623(8171374) done Nov 05 11:21:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_542(8171178) WEXITSTATUS 0 Nov 05 11:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_542(8171178) done Nov 05 11:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_528(8171164) WEXITSTATUS 0 Nov 05 11:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_528(8171164) done Nov 05 11:21:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_576(8171327) WEXITSTATUS 0 Nov 05 11:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_576(8171327) done Nov 05 11:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_499(8171135) WEXITSTATUS 0 Nov 05 11:21:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_499(8171135) done Nov 05 11:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_503(8171139) WEXITSTATUS 0 Nov 05 11:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_503(8171139) done Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:21:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_564(8171315) WEXITSTATUS 0 Nov 05 11:21:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_564(8171315) done Nov 05 11:21:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8167247 Nov 05 11:21:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3957(8169338) WEXITSTATUS 0 Nov 05 11:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3957(8169338) done Nov 05 11:21:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7614(8167059) WEXITSTATUS 1 Nov 05 11:21:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7614(8167059) done Nov 05 11:21:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_533(8171169) WEXITSTATUS 0 Nov 05 11:21:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_533(8171169) done Nov 05 11:21:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_561(8171312) WEXITSTATUS 0 Nov 05 11:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_561(8171312) done Nov 05 11:21:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_548(8171184) WEXITSTATUS 0 Nov 05 11:22:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_548(8171184) done Nov 05 11:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172594 InitPrio=51301 usec=3420 Nov 05 11:22:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_643(8171394) WEXITSTATUS 0 Nov 05 11:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_643(8171394) done Nov 05 11:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7915(8170358) WEXITSTATUS 1 Nov 05 11:22:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7915(8170358) done Nov 05 11:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_631(8171382) WEXITSTATUS 0 Nov 05 11:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_631(8171382) done Nov 05 11:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7381(8166133) WEXITSTATUS 1 Nov 05 11:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7381(8166133) done Nov 05 11:22:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1120(8131612) WEXITSTATUS 0 Nov 05 11:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1120(8131612) done Nov 05 11:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3634(8137001) WEXITSTATUS 1 Nov 05 11:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3634(8137001) done Nov 05 11:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_517(8171153) WEXITSTATUS 0 Nov 05 11:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_517(8171153) done Nov 05 11:22:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_531(8171167) WEXITSTATUS 0 Nov 05 11:22:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_531(8171167) done Nov 05 11:22:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_577(8171328) WEXITSTATUS 0 Nov 05 11:22:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_577(8171328) done Nov 05 11:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7888(8170006) WEXITSTATUS 1 Nov 05 11:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7888(8170006) done Nov 05 11:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_518(8171154) WEXITSTATUS 0 Nov 05 11:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_518(8171154) done Nov 05 11:22:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_653(8171404) WEXITSTATUS 0 Nov 05 11:22:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_653(8171404) done Nov 05 11:22:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_624(8171375) WEXITSTATUS 0 Nov 05 11:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_624(8171375) done Nov 05 11:22:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_511(8171147) WEXITSTATUS 0 Nov 05 11:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_511(8171147) done Nov 05 11:22:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_506(8171142) WEXITSTATUS 0 Nov 05 11:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_506(8171142) done Nov 05 11:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_616(8171367) WEXITSTATUS 0 Nov 05 11:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_616(8171367) done Nov 05 11:22:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5485(8160134) WEXITSTATUS 1 Nov 05 11:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5485(8160134) done Nov 05 11:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_591(8171342) WEXITSTATUS 0 Nov 05 11:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_591(8171342) done Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_505(8171141) WEXITSTATUS 0 Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_505(8171141) done Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_646(8171397) WEXITSTATUS 0 Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_646(8171397) done Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_551(8171187) WEXITSTATUS 0 Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_551(8171187) done Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167248 WEXITSTATUS 0 Nov 05 11:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167248 done Nov 05 11:22:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_521(8171157) WEXITSTATUS 0 Nov 05 11:22:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_521(8171157) done Nov 05 11:22:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_622(8171373) WEXITSTATUS 0 Nov 05 11:22:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_622(8171373) done Nov 05 11:22:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172595 InitPrio=61795 usec=8197 Nov 05 11:22:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_615(8171366) WEXITSTATUS 0 Nov 05 11:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_615(8171366) done Nov 05 11:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_444(8170965) WEXITSTATUS 0 Nov 05 11:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_444(8170965) done Nov 05 11:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_574(8171325) WEXITSTATUS 0 Nov 05 11:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_574(8171325) done Nov 05 11:22:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172596 InitPrio=51301 usec=3676 Nov 05 11:22:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_578(8171329) WEXITSTATUS 0 Nov 05 11:22:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_578(8171329) done Nov 05 11:22:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3654(8137790) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:22:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6050(8163073) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:22:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2419(8165659) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:22:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2710(8166545) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:22:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_2754(8166781) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:22:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7738(8168841) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:22:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172597 InitPrio=48211 usec=3675 Nov 05 11:22:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:22:48 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8172598 NodeList=(null) usec=23940 Nov 05 11:22:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3654(8137790) Nov 05 11:22:49 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7738(8168841) Nov 05 11:22:49 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6050(8163073) Nov 05 11:22:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2419(8165659) Nov 05 11:22:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4007(8172570) WEXITSTATUS 0 Nov 05 11:22:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4007(8172570) done Nov 05 11:22:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_632(8171383) WEXITSTATUS 0 Nov 05 11:22:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_632(8171383) done Nov 05 11:22:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2710(8166545) Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_2754(8166781) Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172598 in devel on r806u25n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172594 in pi_poland on r911u05n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_513(8172599) in pi_ohern on r801u32n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_514(8172600) in pi_ohern on r801u32n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_515(8172601) in pi_ohern on r801u32n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_516(8172602) in pi_ohern on r801u32n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_517(8172603) in pi_ohern on r917u13n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_518(8172604) in pi_ohern on r917u13n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_519(8172605) in pi_ohern on r917u13n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_520(8172606) in pi_ohern on r917u13n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_521(8172607) in pi_ohern on r917u13n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_522(8172608) in pi_ohern on r917u13n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_523(8172609) in pi_ohern on r917u13n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_524(8172610) in pi_ohern on r917u13n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_525(8172611) in pi_ohern on r917u13n02 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_526(8172612) in pi_ohern on r917u13n03 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_527(8172613) in pi_ohern on r917u13n03 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_528(8172614) in pi_ohern on r917u13n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_529(8172615) in pi_ohern on r917u13n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_530(8172616) in pi_ohern on r917u13n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_531(8172617) in pi_ohern on r917u13n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_532(8172618) in pi_ohern on r917u13n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_533(8172619) in pi_ohern on r917u13n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_534(8172620) in pi_ohern on r917u29n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_535(8172621) in pi_ohern on r917u29n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_536(8172622) in pi_ohern on r917u29n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_537(8172623) in pi_ohern on r917u29n03 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_538(8172624) in pi_ohern on r917u29n03 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_539(8172625) in pi_ohern on r917u29n03 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_540(8172626) in pi_ohern on r917u29n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_541(8172627) in pi_ohern on r917u29n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_542(8172628) in pi_ohern on r917u29n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_543(8172629) in pi_ohern on r917u29n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_544(8172630) in pi_ohern on r917u29n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_545(8172631) in pi_ohern on r917u29n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_546(8172632) in pi_ohern on r917u29n04 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_547(8172633) in pi_ohern on r815u03n11 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_548(8172634) in pi_ohern on r815u03n11 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_549(8172635) in pi_ohern on r815u03n11 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_550(8172636) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_551(8172637) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_552(8172638) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_553(8172639) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_554(8172640) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_555(8172641) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_556(8172642) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_557(8172643) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_558(8172644) in pi_ohern on r815u03n12 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_559(8172645) in pi_ohern on r807u22n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_560(8172646) in pi_ohern on r807u24n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_561(8172647) in pi_ohern on r807u24n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_562(8172648) in pi_ohern on r903u12n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_563(8172649) in pi_ohern on r903u12n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_564(8172650) in pi_ohern on r903u12n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_565(8172651) in pi_ohern on r903u12n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_566(8172652) in pi_ohern on r903u12n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_567(8172653) in pi_ohern on r903u26n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_568(8172654) in pi_ohern on r903u26n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_569(8172655) in pi_ohern on r903u26n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_570(8172656) in pi_ohern on r903u26n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_571(8172657) in pi_ohern on r903u26n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_572(8172658) in pi_ohern on r913u32n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_573(8172659) in pi_ohern on r913u32n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_574(8172660) in pi_ohern on r913u32n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_575(8172661) in pi_ohern on r913u32n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_576(8172662) in pi_ohern on r913u35n01 Nov 05 11:22:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_577(8172663) in pi_ohern on r913u35n01 Nov 05 11:22:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172595 in day on r805u09n02 Nov 05 11:22:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4009(8172664) in gpu on r903u34n01 Nov 05 11:22:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4010(8172665) in gpu on r808u14n01 Nov 05 11:22:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4011(8172666) in gpu on r808u17n01 Nov 05 11:22:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172436 in day on r914u05n02 Nov 05 11:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144147 WEXITSTATUS 0 Nov 05 11:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144147 done Nov 05 11:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_609(8171360) WEXITSTATUS 0 Nov 05 11:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_609(8171360) done Nov 05 11:22:54 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144172 NodeList=r903u04n02,r917u07n01 #CPUs=96 Partition=scavenge Nov 05 11:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_520(8171156) WEXITSTATUS 0 Nov 05 11:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_520(8171156) done Nov 05 11:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_60(8135759) WEXITSTATUS 0 Nov 05 11:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_60(8135759) done Nov 05 11:22:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172596 NodeList=r911u05n01 #CPUs=36 Partition=pi_poland Nov 05 11:22:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172667 InitPrio=57401 usec=4532 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8081(8172668) in scavenge on r801u30n03 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8082(8172669) in scavenge on r907u01n04 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8083(8172670) in scavenge on r802u09n04 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8084(8172671) in scavenge on r912u05n04 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8085(8172672) in scavenge on r917u10n04 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8086(8172673) in scavenge on r816u23n10 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8087(8172674) in scavenge on r815u03n10 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8088(8172675) in scavenge on r815u15n12 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8089(8172676) in scavenge on r816u03n09 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8090(8172677) in scavenge on r816u03n12 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8091(8172678) in scavenge on r816u15n10 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8092(8172679) in scavenge on r912u32n04 Nov 05 11:23:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8093(8172680) in scavenge on r914u05n04 Nov 05 11:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_566(8171317) WEXITSTATUS 0 Nov 05 11:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_566(8171317) done Nov 05 11:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_575(8171326) WEXITSTATUS 0 Nov 05 11:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_575(8171326) done Nov 05 11:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8025(8172136) WEXITSTATUS 1 Nov 05 11:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8025(8172136) done Nov 05 11:23:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2950(8172681) in scavenge on r816u03n09 Nov 05 11:23:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2951(8172682) in scavenge on r816u03n09 Nov 05 11:23:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_596(8171347) WEXITSTATUS 0 Nov 05 11:23:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_596(8171347) done Nov 05 11:23:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172436 WEXITSTATUS 0 Nov 05 11:23:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172436 done Nov 05 11:23:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172667 uid 20536 Nov 05 11:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4009(8172664) WEXITSTATUS 0 Nov 05 11:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4009(8172664) done Nov 05 11:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_580(8171331) WEXITSTATUS 0 Nov 05 11:23:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_580(8171331) done Nov 05 11:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_605(8171356) WEXITSTATUS 0 Nov 05 11:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_605(8171356) done Nov 05 11:23:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_571(8171322) WEXITSTATUS 0 Nov 05 11:23:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_571(8171322) done Nov 05 11:23:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172683 InitPrio=51301 usec=4615 Nov 05 11:23:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7901(8170108) WEXITSTATUS 1 Nov 05 11:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7901(8170108) done Nov 05 11:23:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172684 InitPrio=57401 usec=4787 Nov 05 11:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144149 WEXITSTATUS 0 Nov 05 11:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144149 done Nov 05 11:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_498(8171134) WEXITSTATUS 0 Nov 05 11:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_498(8171134) done Nov 05 11:23:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_589(8171340) WEXITSTATUS 0 Nov 05 11:23:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_589(8171340) done Nov 05 11:23:27 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172683 NodeList=r911u01n03 #CPUs=36 Partition=pi_poland Nov 05 11:23:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_657(8171521) WEXITSTATUS 0 Nov 05 11:23:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_657(8171521) done Nov 05 11:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_500(8171136) WEXITSTATUS 0 Nov 05 11:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_500(8171136) done Nov 05 11:23:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172684 NodeList=r806u28n01 #CPUs=1 Partition=bigmem Nov 05 11:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_559(8171195) WEXITSTATUS 0 Nov 05 11:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_559(8171195) done Nov 05 11:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_562(8171313) WEXITSTATUS 0 Nov 05 11:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_562(8171313) done Nov 05 11:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_524(8171160) WEXITSTATUS 0 Nov 05 11:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_524(8171160) done Nov 05 11:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_539(8171175) WEXITSTATUS 0 Nov 05 11:23:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_539(8171175) done Nov 05 11:23:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_654(8171518) WEXITSTATUS 0 Nov 05 11:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_654(8171518) done Nov 05 11:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_608(8171359) WEXITSTATUS 0 Nov 05 11:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_608(8171359) done Nov 05 11:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_617(8171368) WEXITSTATUS 0 Nov 05 11:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_617(8171368) done Nov 05 11:23:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7087(8165628) WEXITSTATUS 1 Nov 05 11:23:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7087(8165628) done Nov 05 11:23:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:23:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:23:46 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_8033(8172279) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:23:46 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_8048(8172429) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:23:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_8033(8172279) Nov 05 11:23:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_8048(8172429) Nov 05 11:23:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172597 NodeList=r806u37n01 #CPUs=16 Partition=gpu Nov 05 11:23:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167601 WEXITSTATUS 0 Nov 05 11:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8167601 done Nov 05 11:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_602(8171353) WEXITSTATUS 0 Nov 05 11:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_602(8171353) done Nov 05 11:23:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_573(8171324) WEXITSTATUS 0 Nov 05 11:23:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_573(8171324) done Nov 05 11:23:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_563(8171314) WEXITSTATUS 0 Nov 05 11:23:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_563(8171314) done Nov 05 11:23:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:23:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8172685 NodeList=(null) usec=26150 Nov 05 11:23:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8172686 NodeList=(null) usec=9221 Nov 05 11:23:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172685 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 11:23:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_572(8171323) WEXITSTATUS 0 Nov 05 11:24:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_572(8171323) done Nov 05 11:24:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_512(8171148) WEXITSTATUS 0 Nov 05 11:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_512(8171148) done Nov 05 11:24:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4477(8155850) WEXITSTATUS 1 Nov 05 11:24:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4477(8155850) done Nov 05 11:24:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_586(8171337) WEXITSTATUS 0 Nov 05 11:24:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_586(8171337) done Nov 05 11:24:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8172687 NodeList=(null) usec=23117 Nov 05 11:24:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172687 NodeList=r908u33n01 #CPUs=1 Partition=admintest Nov 05 11:24:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_525(8171161) WEXITSTATUS 0 Nov 05 11:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_525(8171161) done Nov 05 11:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6999(8165393) WEXITSTATUS 1 Nov 05 11:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6999(8165393) done Nov 05 11:24:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_603(8171354) WEXITSTATUS 0 Nov 05 11:24:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_603(8171354) done Nov 05 11:24:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172687 WEXITSTATUS 0 Nov 05 11:24:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172687 done Nov 05 11:24:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172597 WEXITSTATUS 1 Nov 05 11:24:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172597 done Nov 05 11:24:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:28 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8172688 NodeList=(null) usec=20279 Nov 05 11:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_658(8171522) WEXITSTATUS 0 Nov 05 11:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_658(8171522) done Nov 05 11:24:31 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172688 NodeList=r903u01n03 #CPUs=1 Partition=admintest Nov 05 11:24:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7868(8169787) WEXITSTATUS 1 Nov 05 11:24:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7868(8169787) done Nov 05 11:24:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_611(8171362) WEXITSTATUS 0 Nov 05 11:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_611(8171362) done Nov 05 11:24:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6701(8164806) WEXITSTATUS 1 Nov 05 11:24:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6701(8164806) done Nov 05 11:24:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_640(8171391) WEXITSTATUS 0 Nov 05 11:24:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_640(8171391) done Nov 05 11:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_584(8171335) WEXITSTATUS 0 Nov 05 11:24:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_584(8171335) done Nov 05 11:24:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_607(8171358) WEXITSTATUS 0 Nov 05 11:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_607(8171358) done Nov 05 11:24:42 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 11:24:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_648(8171399) WEXITSTATUS 0 Nov 05 11:24:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_648(8171399) done Nov 05 11:24:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172689 InitPrio=34224 usec=4065 Nov 05 11:24:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7651306_4(7651457) Nov 05 11:24:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172689 NodeList=r913u13n03 #CPUs=4 Partition=day Nov 05 11:24:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172686 NodeList=r806u33n01 #CPUs=8 Partition=gpu_devel Nov 05 11:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_599(8171350) WEXITSTATUS 0 Nov 05 11:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_599(8171350) done Nov 05 11:24:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8083(8172670) WEXITSTATUS 1 Nov 05 11:24:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8083(8172670) done Nov 05 11:24:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_597(8171348) WEXITSTATUS 0 Nov 05 11:24:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_597(8171348) done Nov 05 11:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6858(8165071) WEXITSTATUS 1 Nov 05 11:25:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6858(8165071) done Nov 05 11:25:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_633(8171384) WEXITSTATUS 0 Nov 05 11:25:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_633(8171384) done Nov 05 11:25:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7659(8167611) WEXITSTATUS 1 Nov 05 11:25:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7659(8167611) done Nov 05 11:25:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_569(8171320) WEXITSTATUS 0 Nov 05 11:25:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_569(8171320) done Nov 05 11:25:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7847(8169616) WEXITSTATUS 1 Nov 05 11:25:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7847(8169616) done Nov 05 11:25:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_595(8171346) WEXITSTATUS 0 Nov 05 11:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_595(8171346) done Nov 05 11:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_581(8171332) WEXITSTATUS 0 Nov 05 11:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_581(8171332) done Nov 05 11:25:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_661(8171525) WEXITSTATUS 0 Nov 05 11:25:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_661(8171525) done Nov 05 11:25:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_606(8171357) WEXITSTATUS 0 Nov 05 11:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_606(8171357) done Nov 05 11:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_613(8171364) WEXITSTATUS 0 Nov 05 11:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_613(8171364) done Nov 05 11:25:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4003(8172413) WEXITSTATUS 0 Nov 05 11:25:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4003(8172413) done Nov 05 11:25:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6779(8164931) WEXITSTATUS 1 Nov 05 11:25:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6779(8164931) done Nov 05 11:25:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_656(8171520) WEXITSTATUS 0 Nov 05 11:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_656(8171520) done Nov 05 11:25:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7806(8169349) WEXITSTATUS 1 Nov 05 11:25:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7806(8169349) done Nov 05 11:25:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_578(8172690) in pi_ohern on r801u32n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_579(8172691) in pi_ohern on r801u32n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_580(8172692) in pi_ohern on r801u32n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_581(8172693) in pi_ohern on r801u32n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_582(8172694) in pi_ohern on r917u13n01 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_583(8172695) in pi_ohern on r917u13n01 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_584(8172696) in pi_ohern on r917u13n01 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_585(8172697) in pi_ohern on r917u13n01 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_586(8172698) in pi_ohern on r917u13n01 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_587(8172699) in pi_ohern on r917u13n01 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_588(8172700) in pi_ohern on r917u13n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_589(8172701) in pi_ohern on r917u13n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_590(8172702) in pi_ohern on r917u13n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_591(8172703) in pi_ohern on r917u13n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_592(8172704) in pi_ohern on r917u13n02 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_593(8172705) in pi_ohern on r917u13n03 Nov 05 11:25:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_594(8172706) in pi_ohern on r917u29n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_595(8172707) in pi_ohern on r917u29n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_596(8172708) in pi_ohern on r917u29n02 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_597(8172709) in pi_ohern on r917u29n03 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_598(8172710) in pi_ohern on r917u29n03 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_599(8172711) in pi_ohern on r917u29n03 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_600(8172712) in pi_ohern on r917u29n03 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_601(8172713) in pi_ohern on r917u29n03 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_602(8172714) in pi_ohern on r917u29n03 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_603(8172715) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_604(8172716) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_605(8172717) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_606(8172718) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_607(8172719) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_608(8172720) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_609(8172721) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_610(8172722) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_611(8172723) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_612(8172724) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_613(8172725) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_614(8172726) in pi_ohern on r917u29n04 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_615(8172727) in pi_ohern on r815u03n11 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_616(8172728) in pi_ohern on r807u24n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_617(8172729) in pi_ohern on r903u26n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_618(8172730) in pi_ohern on r903u26n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_619(8172731) in pi_ohern on r911u32n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_620(8172732) in pi_ohern on r913u32n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4012(8172733) in gpu on r808u14n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4013(8172734) in gpu on r903u34n01 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172510 in day on r914u05n02 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r914u05n02 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_209(8114475) Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_659(8171523) WEXITSTATUS 0 Nov 05 11:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_659(8171523) done Nov 05 11:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144141 WEXITSTATUS 0 Nov 05 11:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144141 done Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8094(8172735) in scavenge on r802u09n04 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8095(8172736) in scavenge on r911u09n01 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8096(8172737) in scavenge on r917u07n04 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8097(8172738) in scavenge on r801u06n03 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8098(8172739) in scavenge on r801u09n03 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8099(8172740) in scavenge on r801u23n03 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8100(8172741) in scavenge on r816u03n12 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8101(8172742) in scavenge on r816u29n08 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8102(8172743) in scavenge on r806u09n03 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8103(8172744) in scavenge on r913u09n01 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8104(8172745) in scavenge on r909u10n02 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8105(8172746) in scavenge on r909u10n04 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8106(8172747) in scavenge on r815u09n07 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8107(8172748) in scavenge on r806u37n01 Nov 05 11:25:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8108(8172749) in scavenge on r806u37n01 Nov 05 11:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7574(8166812) WEXITSTATUS 1 Nov 05 11:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7574(8166812) done Nov 05 11:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_582(8171333) WEXITSTATUS 0 Nov 05 11:26:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_582(8171333) done Nov 05 11:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4005(8172486) WEXITSTATUS 0 Nov 05 11:26:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4005(8172486) done Nov 05 11:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172510 WEXITSTATUS 0 Nov 05 11:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172510 done Nov 05 11:26:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172750 InitPrio=16374 usec=7554 Nov 05 11:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_666(8171530) WEXITSTATUS 0 Nov 05 11:26:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_666(8171530) done Nov 05 11:26:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_590(8171341) WEXITSTATUS 0 Nov 05 11:26:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_590(8171341) done Nov 05 11:26:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_594(8171345) WEXITSTATUS 0 Nov 05 11:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_594(8171345) done Nov 05 11:26:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_504(8171140) WEXITSTATUS 0 Nov 05 11:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_504(8171140) done Nov 05 11:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_592(8171343) WEXITSTATUS 0 Nov 05 11:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_592(8171343) done Nov 05 11:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_655(8171519) WEXITSTATUS 0 Nov 05 11:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_655(8171519) done Nov 05 11:26:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172511 WEXITSTATUS 0 Nov 05 11:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172511 done Nov 05 11:26:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7775(8169114) WEXITSTATUS 1 Nov 05 11:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7775(8169114) done Nov 05 11:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_630(8171381) WEXITSTATUS 0 Nov 05 11:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_630(8171381) done Nov 05 11:26:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_634(8171385) WEXITSTATUS 0 Nov 05 11:26:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_634(8171385) done Nov 05 11:26:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:26:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7651306_4(7651457) NodeList=r902u01n01 #CPUs=1 Partition=scavenge_gpu Nov 05 11:26:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144156 NodeList=r911u05n03,r912u05n01 #CPUs=72 Partition=scavenge Nov 05 11:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1024(7573193) WEXITSTATUS 0 Nov 05 11:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1024(7573193) done Nov 05 11:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_526(8171162) WEXITSTATUS 0 Nov 05 11:26:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_526(8171162) done Nov 05 11:26:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_664(8171528) WEXITSTATUS 0 Nov 05 11:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_664(8171528) done Nov 05 11:26:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172751 InitPrio=55324 usec=9703 Nov 05 11:26:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172751 NodeList=r914u30n01 #CPUs=1 Partition=education_gpu Nov 05 11:26:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_598(8171349) WEXITSTATUS 0 Nov 05 11:27:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_598(8171349) done Nov 05 11:27:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_549(8171185) WEXITSTATUS 0 Nov 05 11:27:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_549(8171185) done Nov 05 11:27:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7768(8169041) WEXITSTATUS 1 Nov 05 11:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7768(8169041) done Nov 05 11:27:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_667(8171531) WEXITSTATUS 0 Nov 05 11:27:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_667(8171531) done Nov 05 11:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7867(8169786) WEXITSTATUS 1 Nov 05 11:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7867(8169786) done Nov 05 11:27:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_663(8171527) WEXITSTATUS 0 Nov 05 11:27:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_663(8171527) done Nov 05 11:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_610(8171361) WEXITSTATUS 0 Nov 05 11:27:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_610(8171361) done Nov 05 11:27:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_570(8171321) WEXITSTATUS 0 Nov 05 11:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_570(8171321) done Nov 05 11:27:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_583(8171334) WEXITSTATUS 0 Nov 05 11:27:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_583(8171334) done Nov 05 11:27:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:27:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7564(8166779) WEXITSTATUS 1 Nov 05 11:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7564(8166779) done Nov 05 11:27:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_604(8171355) WEXITSTATUS 0 Nov 05 11:27:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_604(8171355) done Nov 05 11:27:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_565(8171316) WEXITSTATUS 0 Nov 05 11:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_565(8171316) done Nov 05 11:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7921(8170364) WEXITSTATUS 1 Nov 05 11:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7921(8170364) done Nov 05 11:27:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_585(8171336) WEXITSTATUS 0 Nov 05 11:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_585(8171336) done Nov 05 11:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_41(8165421) WEXITSTATUS 0 Nov 05 11:27:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_41(8165421) done Nov 05 11:27:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:27:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172514 NodeList=r815u03n[02-03,08],r815u09n[01,04,08],r815u15n08,r815u23n05,r815u29n[02,04-05],r815u35n01 #CPUs=288 Partition=day Nov 05 11:27:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172750 NodeList=r806u02n02 #CPUs=1 Partition=day Nov 05 11:27:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2952(8172752) NodeList=r914u05n02 #CPUs=1 Partition=day Nov 05 11:27:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172753 InitPrio=16374 usec=4112 Nov 05 11:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172750 WEXITSTATUS 0 Nov 05 11:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172750 done Nov 05 11:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_530(8171166) WEXITSTATUS 0 Nov 05 11:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_530(8171166) done Nov 05 11:27:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4012(8172733) WEXITSTATUS 0 Nov 05 11:28:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4012(8172733) done Nov 05 11:28:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7688(8168461) WEXITSTATUS 1 Nov 05 11:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7688(8168461) done Nov 05 11:28:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_612(8171363) WEXITSTATUS 0 Nov 05 11:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_612(8171363) done Nov 05 11:28:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154708 WEXITSTATUS 0 Nov 05 11:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154708 done Nov 05 11:28:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_665(8171529) WEXITSTATUS 0 Nov 05 11:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_665(8171529) done Nov 05 11:28:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_587(8171338) WEXITSTATUS 0 Nov 05 11:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_587(8171338) done Nov 05 11:28:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:28:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172754 InitPrio=44927 usec=7263 Nov 05 11:28:34 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172754 NodeList=r806u25n01 #CPUs=1 Partition=devel Nov 05 11:28:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_621(8172755) in pi_ohern on r917u13n01 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_622(8172756) in pi_ohern on r917u13n01 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_623(8172757) in pi_ohern on r917u13n01 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_624(8172758) in pi_ohern on r917u13n01 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_625(8172759) in pi_ohern on r917u13n02 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_626(8172760) in pi_ohern on r917u13n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_627(8172761) in pi_ohern on r917u13n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_628(8172762) in pi_ohern on r917u13n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_629(8172763) in pi_ohern on r917u13n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_630(8172764) in pi_ohern on r917u13n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_631(8172765) in pi_ohern on r917u29n02 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_632(8172766) in pi_ohern on r917u29n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_633(8172767) in pi_ohern on r917u29n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_634(8172768) in pi_ohern on r917u29n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_635(8172769) in pi_ohern on r917u29n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_636(8172770) in pi_ohern on r917u29n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_637(8172771) in pi_ohern on r917u29n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_638(8172772) in pi_ohern on r917u29n03 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_639(8172773) in pi_ohern on r917u29n04 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_640(8172774) in pi_ohern on r917u29n04 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_641(8172775) in pi_ohern on r917u29n04 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_642(8172776) in pi_ohern on r917u29n04 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_643(8172777) in pi_ohern on r917u29n04 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_644(8172778) in pi_ohern on r807u22n01 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_645(8172779) in pi_ohern on r903u12n01 Nov 05 11:28:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_646(8172780) in pi_ohern on r903u12n01 Nov 05 11:28:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4014(8172781) in gpu on r808u14n01 Nov 05 11:28:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4015(8172782) in gpu on r4513u26n01 Nov 05 11:28:40 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2953(8172783) in day on r919u01n01 Nov 05 11:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_660(8171524) WEXITSTATUS 0 Nov 05 11:28:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_660(8171524) done Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8109(8172784) in scavenge on r802u04n03 Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8110(8172785) in scavenge on r816u15n12 Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8111(8172786) in scavenge on r902u01n01 Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8112(8172787) in scavenge on r909u10n02 Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8113(8172788) in scavenge on r914u01n03 Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8114(8172789) in scavenge on r805u04n02 Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8115(8172790) in scavenge on r805u09n01 Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8116(8172791) in scavenge on r918u05n04 Nov 05 11:28:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8117(8172792) in scavenge on r918u05n04 Nov 05 11:28:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_773(8115042) in scavenge on r919u01n01 Nov 05 11:28:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_906(7573075) in scavenge on r919u09n01 Nov 05 11:28:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_222(8114488) in scavenge on r815u35n04 Nov 05 11:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_662(8171526) WEXITSTATUS 0 Nov 05 11:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_662(8171526) done Nov 05 11:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8074(8172582) WEXITSTATUS 1 Nov 05 11:28:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8074(8172582) done Nov 05 11:28:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8053(8172491) WEXITSTATUS 1 Nov 05 11:28:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8053(8172491) done Nov 05 11:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170111 WEXITSTATUS 1 Nov 05 11:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170111 done Nov 05 11:29:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172793 InitPrio=56564 usec=5828 Nov 05 11:29:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172793 NodeList=r913u01n03 #CPUs=1 Partition=day Nov 05 11:29:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8158658 uid 17360 Nov 05 11:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7526(8166569) WEXITSTATUS 1 Nov 05 11:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7526(8166569) done Nov 05 11:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_507(8171143) WEXITSTATUS 0 Nov 05 11:29:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_507(8171143) done Nov 05 11:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_588(8171339) WEXITSTATUS 0 Nov 05 11:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_588(8171339) done Nov 05 11:29:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172794 InitPrio=64418 usec=7268 Nov 05 11:29:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172794 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:29:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_569(8114836) WEXITSTATUS 0 Nov 05 11:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_569(8114836) done Nov 05 11:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1055(7572021) WEXITSTATUS 0 Nov 05 11:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1055(7572021) done Nov 05 11:30:17 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8170110 Nov 05 11:30:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_568(8171319) WEXITSTATUS 0 Nov 05 11:30:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_568(8171319) done Nov 05 11:30:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8031(8172142) WEXITSTATUS 1 Nov 05 11:30:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8031(8172142) done Nov 05 11:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7957(8170791) WEXITSTATUS 1 Nov 05 11:30:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7957(8170791) done Nov 05 11:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8065(8172573) WEXITSTATUS 1 Nov 05 11:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8065(8172573) done Nov 05 11:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_560(8171311) WEXITSTATUS 0 Nov 05 11:30:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_560(8171311) done Nov 05 11:30:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r807u03n02 #CPUs=1 Partition=day Nov 05 11:30:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r807u03n02 #CPUs=1 Partition=day Nov 05 11:30:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172753 NodeList=r909u07n02 #CPUs=32 Partition=day Nov 05 11:30:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6327612 NodeList=r807u03n02 #CPUs=1 Partition=day Nov 05 11:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7913(8170212) WEXITSTATUS 1 Nov 05 11:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7913(8170212) done Nov 05 11:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_601(8171352) WEXITSTATUS 0 Nov 05 11:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8168792_601(8171352) done Nov 05 11:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1458(8137918) WEXITSTATUS 0 Nov 05 11:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1458(8137918) done Nov 05 11:31:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172795 InitPrio=28349 usec=7093 Nov 05 11:31:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7635(8167085) WEXITSTATUS 1 Nov 05 11:31:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7635(8167085) done Nov 05 11:31:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144139 WEXITSTATUS 0 Nov 05 11:31:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144139 done Nov 05 11:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 11:31:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 11:31:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 11:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3997(8172122) WEXITSTATUS 0 Nov 05 11:31:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3997(8172122) done Nov 05 11:31:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:31:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_647(8172796) in pi_ohern on r801u32n02 Nov 05 11:31:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_648(8172797) in pi_ohern on r917u13n01 Nov 05 11:31:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_649(8172798) in pi_ohern on r917u13n02 Nov 05 11:31:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_650(8172799) in pi_ohern on r917u13n02 Nov 05 11:31:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_651(8172800) in pi_ohern on r917u13n02 Nov 05 11:31:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_652(8172801) in pi_ohern on r917u29n03 Nov 05 11:31:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_653(8172802) in pi_ohern on r917u29n04 Nov 05 11:31:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172795 in gpu on r808u08n01 Nov 05 11:31:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8118(8172803) in scavenge on r903u01n02 Nov 05 11:31:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8119(8172804) in scavenge on r916u04n01 Nov 05 11:31:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8120(8172805) in scavenge on r815u03n10 Nov 05 11:31:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8121(8172806) in scavenge on r815u23n08 Nov 05 11:31:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8122(8172807) in scavenge on r816u23n11 Nov 05 11:31:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8123(8172808) in scavenge on r907u28n01 Nov 05 11:31:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8124(8172809) in scavenge on r807u03n02 Nov 05 11:31:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8125(8172810) in scavenge on r807u03n02 Nov 05 11:31:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_878(8115147) in scavenge on r805u11n02 Nov 05 11:31:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_900(8115169) in scavenge on r805u23n01 Nov 05 11:31:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_756(7571721) in scavenge on r806u06n04 Nov 05 11:31:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_876(7573045) in scavenge on r806u11n03 Nov 05 11:31:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_913(7573082) in scavenge on r806u11n04 Nov 05 11:31:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1165(7570827) in scavenge on r816u09n08 Nov 05 11:31:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_481(8114748) in scavenge on r815u03n07 Nov 05 11:31:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_14(8114280) in scavenge on r815u09n02 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2954(8172811) in scavenge on r816u29n11 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_20 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_16 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_17 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_18 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_14 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_19 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_24 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_29 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_22 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_23 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_25 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_21 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_26 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_31 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_32 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_30 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_39 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_28 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_33 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_37 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_38 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_27 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_36 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_42 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_35 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_40 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_45 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_46 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_43 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_48 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_41 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_50 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_55 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_44 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_49 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_34 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_47 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_51 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_56 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_57 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_58 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_60 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_53 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_54 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_67 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_59 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_61 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_66 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_52 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_65 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_70 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_63 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_71 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_72 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_62 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_74 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_69 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_77 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_79 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_80 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_64 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_73 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_78 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_81 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_68 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_75 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_83 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_82 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_85 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_87 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_84 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_76 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_86 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_88 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_89 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_2 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_91 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_92 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_1 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_90 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_94 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_3 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_4 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_5 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_95 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_12 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_8 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_93 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_9 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_7 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_11 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_10 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_15 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_13 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_6 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_96 uid 23340 Nov 05 11:31:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_14 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_16 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_17 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_18 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_20 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_21 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_23 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_24 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_26 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_22 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_19 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_25 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_27 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_28 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_29 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_30 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_31 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_32 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_33 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_39 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_36 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_34 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_35 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_37 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_40 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_38 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_42 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_43 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_46 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_44 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_48 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_49 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_41 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_47 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_45 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_50 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_51 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_52 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_53 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_54 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_55 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_56 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_57 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_58 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_59 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_60 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_61 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_62 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_63 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_64 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_68 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_69 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_67 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_65 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_66 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_70 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_71 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_72 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_73 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_74 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_75 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_76 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_77 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_78 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_79 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_80 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_81 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_82 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_83 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_84 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_85 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_86 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_87 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_88 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_93 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_90 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_91 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_96 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_89 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_94 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_92 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_95 uid 23340 Nov 05 11:31:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_[10-96] uid 23340 Nov 05 11:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172753 WEXITSTATUS 1 Nov 05 11:31:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172753 done Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_1 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_1 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_1 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_2 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_2 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_2 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_3 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_3 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_3 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_4 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_4 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_4 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_5 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_5 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_5 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_6 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_6 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_6 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_7 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_7 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_7 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_8 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_8 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_8 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_9 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_9 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_9 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_10 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_10 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_10 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_11 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_11 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_11 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_12 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_12 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_12 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_13 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_13 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_13 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_14 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_14 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_14 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_15 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_15 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_15 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_16 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_16 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_16 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_17 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_17 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_17 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_18 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_18 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_18 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_19 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_19 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_19 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_20 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_20 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_20 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_21 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_21 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_21 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_22 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_22 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_22 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_23 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_23 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_23 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_24 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_24 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_24 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_25 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_25 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_25 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_26 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_26 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_26 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_27 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_27 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_27 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_28 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_28 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_28 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_29 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_29 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_29 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_30 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_30 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_30 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_31 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_31 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_31 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_32 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_32 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_32 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_33 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_33 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_33 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_34 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_34 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_34 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_35 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_35 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_35 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_36 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_36 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_36 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_37 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_37 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_37 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_38 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_38 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_38 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_39 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_39 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_39 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_40 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_40 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_40 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_41 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_41 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_41 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_42 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_42 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_42 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_43 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_43 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_43 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_44 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_44 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_44 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_45 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_45 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_45 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_46 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_46 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_46 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_47 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_47 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_47 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_48 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_48 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_48 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_49 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_49 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_49 sig=9 returned: Invalid job id specified Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_50 uid 23340 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_50 Nov 05 11:31:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_50 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_51 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_51 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_51 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_52 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_52 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_52 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_53 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_53 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_53 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_54 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_54 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_54 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_55 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_55 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_55 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_56 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_56 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_56 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_57 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_57 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_57 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_58 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_58 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_58 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_59 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_59 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_59 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_60 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_60 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_60 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_61 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_61 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_61 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_62 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_62 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_62 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_63 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_63 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_63 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_64 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_64 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_64 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_65 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_65 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_65 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_66 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_66 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_66 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_67 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_67 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_67 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_68 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_68 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_68 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_69 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_69 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_69 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_70 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_70 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_70 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_71 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_71 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_71 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_72 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_72 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_72 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_73 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_73 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_73 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_74 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_74 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_74 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_75 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_75 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_75 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_76 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_76 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_76 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_77 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_77 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_77 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_78 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_78 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_78 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_79 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_79 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_79 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_80 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_80 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_80 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_81 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_81 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_81 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_82 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_82 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_82 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_83 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_83 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_83 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_84 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_84 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_84 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_85 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_85 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_85 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_86 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_86 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_86 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_87 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_87 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_87 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_88 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_88 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_88 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_89 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_89 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_89 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_90 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_90 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_90 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_91 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_91 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_91 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_92 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_92 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_92 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_93 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_93 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_93 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_94 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_94 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_94 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_95 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_95 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_95 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140415_96 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(6): invalid JobId=8140415_96 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=23340 JobId=8140415_96 sig=9 returned: Invalid job id specified Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_1 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_2 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_3 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_4 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_5 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_6 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_7 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_8 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_9 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_10 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_11 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_12 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_13 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_14 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_15 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_16 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_17 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_18 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_19 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_20 uid 23340 Nov 05 11:31:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_21 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_22 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_23 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_24 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_25 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_26 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_27 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_28 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_29 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_30 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_31 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_32 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_33 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_34 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_35 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_36 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_37 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_38 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_39 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_40 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_41 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_42 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_43 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_44 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_45 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_46 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_47 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_48 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_49 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_50 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_51 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_52 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_53 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_54 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_55 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_56 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_57 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_58 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_59 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_60 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_61 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_62 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_63 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_64 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_65 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_66 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_67 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_68 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_69 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_70 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_71 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_72 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_73 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_74 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_75 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_76 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_77 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_78 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_79 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_80 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_81 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_82 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_83 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_84 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_85 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_86 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_87 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_88 uid 23340 Nov 05 11:31:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_89 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_90 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_91 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_92 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_93 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_94 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_95 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140416_96 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_1 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_2 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_3 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_4 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_5 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_6 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_7 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_8 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_9 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_10 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_11 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_12 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_13 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_14 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_15 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_16 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_17 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_18 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_19 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_20 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_21 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_22 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_23 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_24 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_25 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_26 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_27 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_28 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_29 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_30 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_31 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_32 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_33 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_34 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_35 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_36 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_37 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_38 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_39 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_40 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_41 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_42 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_43 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_44 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_45 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_46 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_47 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_48 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_49 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_50 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_51 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_52 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_53 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_54 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_55 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_56 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_57 uid 23340 Nov 05 11:31:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_58 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_59 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_60 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_61 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_62 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_63 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_64 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_65 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_66 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_67 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_68 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_69 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_70 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_71 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_72 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_73 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_74 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_75 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_76 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_77 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_78 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_79 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_80 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_81 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_82 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_83 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_84 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_85 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_86 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_87 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_88 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_89 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_90 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_91 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_92 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_93 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_94 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_95 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140512_96 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_1 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_2 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_3 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_4 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_5 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_6 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_7 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_8 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_9 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_10 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_11 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_12 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_13 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_14 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_15 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_16 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_17 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_18 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_19 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_20 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_21 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_22 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_23 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_24 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_25 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_26 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_27 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_28 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_29 uid 23340 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4014(8172781) WEXITSTATUS 0 Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4014(8172781) done Nov 05 11:31:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_30 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_31 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_32 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_33 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_34 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_35 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_36 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_37 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_38 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_39 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_40 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_41 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_42 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_43 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_44 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_45 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_46 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_47 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_48 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_49 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_50 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_51 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_52 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_53 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_54 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_55 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_56 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_57 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_58 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_59 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_60 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_61 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_62 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_63 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_64 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_65 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_66 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_67 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_68 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_69 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_70 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_71 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_72 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_73 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_74 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_75 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_76 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_77 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_78 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_79 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_80 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_81 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_82 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_83 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_84 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_85 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_86 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_87 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_88 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_89 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_90 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_91 uid 23340 Nov 05 11:31:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_92 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_93 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_94 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_95 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140608_96 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_1 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_2 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_3 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_4 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_5 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_6 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_7 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_8 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_9 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_10 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_11 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_12 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_13 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_14 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_15 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_16 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_17 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_18 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_19 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_20 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_21 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_22 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_23 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_24 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_25 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_26 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_27 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_28 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_29 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_30 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_31 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_32 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_33 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_34 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_35 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_36 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_37 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_38 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_39 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_40 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_41 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_42 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_43 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_44 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_45 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_46 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_47 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_48 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_49 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_50 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_51 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_52 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_53 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_54 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_55 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_56 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_57 uid 23340 Nov 05 11:31:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_58 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_59 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_60 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_61 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_62 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_63 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_64 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_65 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_66 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_67 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_68 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_69 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_70 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_71 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_72 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_73 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_74 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_75 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_76 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_77 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_78 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_79 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_80 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_81 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_82 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_83 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_84 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_85 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_86 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_87 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_88 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_89 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_90 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_91 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_92 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_93 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_94 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_95 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140704_96 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_1 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_2 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_3 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_4 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_5 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_6 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_7 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_8 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_9 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_10 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_11 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_12 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_13 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_14 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_15 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_16 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_17 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_18 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_19 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_20 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_21 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_22 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_23 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_24 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_25 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_26 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_27 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_28 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_29 uid 23340 Nov 05 11:31:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_30 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_31 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_32 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_33 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_34 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_35 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_36 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_37 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_38 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_39 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172795 uid 25715 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_40 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_41 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_42 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_43 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_44 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_45 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_46 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_47 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_48 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_49 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_50 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_51 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_52 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_53 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_54 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_55 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_56 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_57 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_58 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_59 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_60 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_61 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_62 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_63 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_64 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_65 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_66 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_67 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_68 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_69 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_70 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_71 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_72 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_73 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_74 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_75 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_76 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_77 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_78 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_79 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_80 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_81 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_82 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_83 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_84 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_85 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_86 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_87 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_88 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_89 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_90 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_91 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_92 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_93 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_94 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_95 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140800_96 uid 23340 Nov 05 11:31:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_1 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_2 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_3 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_4 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_5 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_6 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_7 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_8 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_9 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_10 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_11 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_12 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_13 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_14 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_15 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_16 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_17 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_18 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_19 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_20 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_21 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_22 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_23 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_24 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_25 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_26 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_27 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_28 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_29 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_30 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_31 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_32 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_33 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_34 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_35 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_36 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_37 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_38 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_39 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_40 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_41 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_42 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_43 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_44 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_45 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_46 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_47 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_48 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_49 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_50 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_51 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_52 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_53 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_54 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_55 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_56 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_57 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_58 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_59 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_60 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_61 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_62 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_63 uid 23340 Nov 05 11:31:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_64 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_65 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7626(8167071) WEXITSTATUS 1 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7626(8167071) done Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_66 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_67 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_68 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_69 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_70 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_71 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_72 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_73 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_74 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_75 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_76 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_77 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_78 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_79 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_80 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_81 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_82 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_83 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_84 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_85 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_86 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_87 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_88 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_89 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_90 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_91 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_92 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_93 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_94 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_95 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140896_96 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_1 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_2 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_3 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_4 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_5 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_6 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_7 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_8 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_9 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_10 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_11 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_12 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_13 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_14 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_15 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_16 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_17 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_18 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_19 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_20 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_21 uid 23340 Nov 05 11:31:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_22 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_23 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_24 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_25 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_26 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_27 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_28 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_29 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_30 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_31 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_32 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_33 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_34 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_35 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_36 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_37 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_38 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_39 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_40 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_41 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_42 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_43 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_44 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_45 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_46 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_47 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_48 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_49 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_50 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_51 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_52 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_53 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_54 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_55 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_56 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_57 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_58 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_59 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_60 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_61 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_62 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_63 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_64 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_65 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_66 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_67 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_68 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_69 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_70 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_71 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_72 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_73 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_74 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_75 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_76 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_77 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_78 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_79 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_80 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_81 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_82 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_83 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_84 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_85 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_86 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_87 uid 23340 Nov 05 11:31:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_88 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_89 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2955(8172812) NodeList=r909u07n02 #CPUs=1 Partition=day Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144158 NodeList=r807u01n04,r816u09n11 #CPUs=84 Partition=scavenge Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_90 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_91 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_92 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_93 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_94 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_95 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8140992_96 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_1 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_2 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_3 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_4 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_5 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_6 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_7 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_8 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_9 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_10 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_11 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_12 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_13 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_14 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_15 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_16 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_17 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_18 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_19 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_20 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_21 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_22 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_23 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_24 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_25 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_26 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_27 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_28 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_29 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_30 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_31 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_32 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_33 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_34 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_35 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_36 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_37 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_38 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_39 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_40 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_41 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_42 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_43 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_44 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_45 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_46 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_47 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_48 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_49 uid 23340 Nov 05 11:31:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_50 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_51 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_52 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_53 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_54 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_55 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_56 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_57 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_58 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_59 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_60 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_61 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_62 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_63 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_64 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_65 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_66 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_67 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_68 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_69 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_70 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_71 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_72 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_73 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_74 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_75 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_76 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_77 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_78 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_79 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_80 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_81 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_82 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_83 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_84 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_85 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_86 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_87 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_88 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_89 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_90 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_91 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_92 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_93 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_94 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_95 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141088_96 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_1 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_2 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_3 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_4 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_5 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_6 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_7 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_8 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_9 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_10 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_11 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_12 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_13 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_14 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_15 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_16 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_17 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_18 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_19 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_20 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_21 uid 23340 Nov 05 11:31:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_22 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_23 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_24 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_25 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_26 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_27 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_28 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_29 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_30 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_31 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_32 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_33 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_34 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_35 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_36 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_37 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_38 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_39 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_40 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_41 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_42 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_43 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_44 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_45 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_46 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_47 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_48 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_49 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_50 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_51 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_52 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_53 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_54 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_55 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_56 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_57 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_58 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_59 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_60 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_61 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_62 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_63 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_64 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_65 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_66 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_67 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_68 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_69 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_70 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_71 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_72 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_73 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_74 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_75 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_76 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_77 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_78 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_79 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_80 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_81 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_82 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_83 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_84 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_85 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_86 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_87 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_88 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_89 uid 23340 Nov 05 11:31:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_90 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_91 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_92 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_93 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_94 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_95 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141184_96 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_1 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_2 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_3 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_4 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_5 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_6 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_7 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_8 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_9 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_10 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_11 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_12 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_13 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_14 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_15 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_16 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_17 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_18 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_19 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_20 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_21 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_22 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_23 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_24 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_25 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_26 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_27 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_28 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_29 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_30 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_31 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_32 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_33 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_34 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_35 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_36 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_37 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_38 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_39 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_40 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_41 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_42 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_43 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_44 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_45 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_46 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_47 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_48 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_49 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_50 uid 23340 Nov 05 11:31:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_51 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_52 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_53 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_54 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_55 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_56 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_57 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_58 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_59 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_60 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_61 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_62 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_63 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_64 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_65 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_66 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_67 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_68 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_69 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_70 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_71 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_72 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_73 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_74 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_75 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_76 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_77 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_78 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_79 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_80 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_81 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_82 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_83 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_84 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_85 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_86 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_87 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_88 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_89 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_90 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_91 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_92 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_93 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_94 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_95 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8141280_96 uid 23340 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172813 InitPrio=13477 usec=4970 Nov 05 11:31:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172814 InitPrio=13477 usec=1771 Nov 05 11:31:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7658(8167610) WEXITSTATUS 1 Nov 05 11:32:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7658(8167610) done Nov 05 11:32:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:32:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172593 WEXITSTATUS 0 Nov 05 11:32:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172593 done Nov 05 11:32:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8087(8172674) WEXITSTATUS 1 Nov 05 11:32:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8087(8172674) done Nov 05 11:32:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172688 WEXITSTATUS 0 Nov 05 11:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172688 done Nov 05 11:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7739(8168842) WEXITSTATUS 1 Nov 05 11:32:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7739(8168842) done Nov 05 11:32:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171991 WEXITSTATUS 130 Nov 05 11:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171991 done Nov 05 11:32:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8105(8172746) WEXITSTATUS 1 Nov 05 11:32:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8105(8172746) done Nov 05 11:32:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8113(8172788) WEXITSTATUS 1 Nov 05 11:32:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8113(8172788) done Nov 05 11:32:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:33:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Invalid account or account/partition combination specified Nov 05 11:33:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7862(8169781) WEXITSTATUS 1 Nov 05 11:33:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7862(8169781) done Nov 05 11:33:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8004(8171678) WEXITSTATUS 1 Nov 05 11:33:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8004(8171678) done Nov 05 11:33:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 11:33:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 11:33:26 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 11:33:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4980(8158242) WEXITSTATUS 1 Nov 05 11:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4980(8158242) done Nov 05 11:33:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4468(8155785) WEXITSTATUS 1 Nov 05 11:33:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4468(8155785) done Nov 05 11:33:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:33:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159208 WEXITSTATUS 0 Nov 05 11:33:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159208 done Nov 05 11:33:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:33:51 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:33:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3652(8137787) WEXITSTATUS 1 Nov 05 11:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3652(8137787) done Nov 05 11:33:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_1(8172815) in pi_balou on r802u30n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_2(8172816) in pi_balou on r802u30n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_3(8172817) in pi_balou on r802u30n04 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_4(8172818) in pi_balou on r802u30n04 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_5(8172819) in pi_balou on r911u09n01 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_6(8172820) in pi_balou on r911u09n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_7(8172821) in pi_balou on r911u09n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_8(8172822) in pi_balou on r911u09n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_9(8172823) in pi_balou on r911u09n03 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_10(8172824) in pi_balou on r911u09n03 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_11(8172825) in pi_balou on r911u09n03 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_12(8172826) in pi_balou on r911u24n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_13(8172827) in pi_balou on r911u28n01 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_14(8172828) in pi_balou on r911u28n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_15(8172829) in pi_balou on r911u28n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_16(8172830) in pi_balou on r911u28n02 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4016(8172831) in gpu on r808u08n01 Nov 05 11:34:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4017(8172832) in gpu on r808u14n01 Nov 05 11:34:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_312(8114579) WEXITSTATUS 0 Nov 05 11:34:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_312(8114579) done Nov 05 11:34:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_664(7572833) WEXITSTATUS 0 Nov 05 11:34:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_664(7572833) done Nov 05 11:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7987(8171302) WEXITSTATUS 1 Nov 05 11:34:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7987(8171302) done Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8126(8172833) in scavenge on r802u06n04 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8127(8172834) in scavenge on r915u32n01 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8128(8172835) in scavenge on r815u03n10 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8129(8172836) in scavenge on r815u15n09 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8130(8172837) in scavenge on r816u23n09 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8131(8172838) in scavenge on r816u35n11 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8132(8172839) in scavenge on r805u11n02 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8133(8172840) in scavenge on r805u23n01 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8134(8172841) in scavenge on r909u07n02 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8135(8172842) in scavenge on r909u07n02 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8136(8172843) in scavenge on r909u07n02 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8137(8172844) in scavenge on r909u10n04 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_814(8115083) in scavenge on r816u15n09 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_38(8114304) in scavenge on r806u14n04 Nov 05 11:34:13 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_70(8114336) in scavenge on r912u32n04 Nov 05 11:34:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172814 in scavenge on r908u33n01 Nov 05 11:34:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7437(8166230) WEXITSTATUS 1 Nov 05 11:34:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7437(8166230) done Nov 05 11:34:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172814 WEXITSTATUS 0 Nov 05 11:34:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172814 done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_12(8172826) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_12(8172826) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_13(8172827) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_13(8172827) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_10(8172824) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_10(8172824) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_9(8172823) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_9(8172823) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_11(8172825) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_11(8172825) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_5(8172819) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_5(8172819) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_1(8172815) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_1(8172815) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_8(8172822) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_8(8172822) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_7(8172821) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_7(8172821) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_6(8172820) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_6(8172820) done Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_2(8172816) WEXITSTATUS 1 Nov 05 11:34:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_2(8172816) done Nov 05 11:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154681 WEXITSTATUS 0 Nov 05 11:34:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154681 done Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_16(8172830) WEXITSTATUS 1 Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_16(8172830) done Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_15(8172829) WEXITSTATUS 1 Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_15(8172829) done Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_14(8172828) WEXITSTATUS 1 Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_14(8172828) done Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_3(8172817) WEXITSTATUS 1 Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_3(8172817) done Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_4(8172818) WEXITSTATUS 1 Nov 05 11:34:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_4(8172818) done Nov 05 11:34:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172845 InitPrio=28349 usec=3733 Nov 05 11:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8084(8172671) WEXITSTATUS 1 Nov 05 11:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8084(8172671) done Nov 05 11:34:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8172845 uid 25715 Nov 05 11:34:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7457(8166293) WEXITSTATUS 1 Nov 05 11:34:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7457(8166293) done Nov 05 11:34:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3940(8145772) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:34:49 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5589(8161552) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:34:49 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8169646 had to be killed Nov 05 11:34:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3940(8145772) Nov 05 11:34:51 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5589(8161552) Nov 05 11:34:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6327612 NodeList=r807u03n02 #CPUs=1 Partition=day Nov 05 11:34:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8123792_2956(8172846) NodeList=r816u09n06 #CPUs=1 Partition=day Nov 05 11:35:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172847 InitPrio=28349 usec=3841 Nov 05 11:35:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7834(8169551) WEXITSTATUS 1 Nov 05 11:35:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7834(8169551) done Nov 05 11:35:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4016(8172831) WEXITSTATUS 0 Nov 05 11:35:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4016(8172831) done Nov 05 11:35:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7785(8169193) WEXITSTATUS 1 Nov 05 11:35:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7785(8169193) done Nov 05 11:35:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:35:52 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:35:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:35:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7200(8165832) WEXITSTATUS 1 Nov 05 11:35:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7200(8165832) done Nov 05 11:35:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5119(8158452) WEXITSTATUS 1 Nov 05 11:36:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5119(8158452) done Nov 05 11:36:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170531 WEXITSTATUS 0 Nov 05 11:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170531 done Nov 05 11:36:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159168 WEXITSTATUS 0 Nov 05 11:36:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159168 done Nov 05 11:36:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_19(8165255) WEXITSTATUS 0 Nov 05 11:36:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_19(8165255) done Nov 05 11:36:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:18 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3278(8134072) Nov 05 11:36:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:36:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4015(8172782) WEXITSTATUS 0 Nov 05 11:36:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4015(8172782) done Nov 05 11:36:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172848 InitPrio=55292 usec=6133 Nov 05 11:36:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8102(8172743) WEXITSTATUS 1 Nov 05 11:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8102(8172743) done Nov 05 11:36:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172849 InitPrio=55292 usec=4139 Nov 05 11:36:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1461(8138138) WEXITSTATUS 0 Nov 05 11:36:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1461(8138138) done Nov 05 11:36:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8082(8172669) WEXITSTATUS 1 Nov 05 11:36:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8082(8172669) done Nov 05 11:36:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172850 InitPrio=55292 usec=4112 Nov 05 11:36:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_17(8172851) in pi_balou on r802u30n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_18(8172852) in pi_balou on r802u30n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_19(8172853) in pi_balou on r802u30n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_20(8172854) in pi_balou on r802u30n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_21(8172855) in pi_balou on r911u09n01 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_22(8172856) in pi_balou on r911u09n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_23(8172857) in pi_balou on r911u09n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_24(8172858) in pi_balou on r911u09n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_25(8172859) in pi_balou on r911u09n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_26(8172860) in pi_balou on r911u09n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_27(8172861) in pi_balou on r911u09n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_28(8172862) in pi_balou on r911u13n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_29(8172863) in pi_balou on r911u13n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_30(8172864) in pi_balou on r911u13n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_31(8172865) in pi_balou on r911u13n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_32(8172866) in pi_balou on r911u13n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_33(8172867) in pi_balou on r911u13n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_34(8172868) in pi_balou on r911u13n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_35(8172869) in pi_balou on r911u13n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_36(8172870) in pi_balou on r911u13n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_37(8172871) in pi_balou on r911u13n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_38(8172872) in pi_balou on r911u13n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_39(8172873) in pi_balou on r911u13n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_40(8172874) in pi_balou on r911u13n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_41(8172875) in pi_balou on r911u13n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_42(8172876) in pi_balou on r911u13n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_43(8172877) in pi_balou on r911u13n03 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_44(8172878) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_45(8172879) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_46(8172880) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_47(8172881) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_48(8172882) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_49(8172883) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_50(8172884) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_51(8172885) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_52(8172886) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_53(8172887) in pi_balou on r911u13n04 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_54(8172888) in pi_balou on r911u24n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_55(8172889) in pi_balou on r911u28n01 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_56(8172890) in pi_balou on r911u28n01 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_57(8172891) in pi_balou on r911u28n01 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_58(8172892) in pi_balou on r911u28n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_59(8172893) in pi_balou on r911u28n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_60(8172894) in pi_balou on r911u28n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_61(8172895) in pi_balou on r911u28n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_62(8172896) in pi_balou on r911u28n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_63(8172897) in pi_balou on r911u28n02 Nov 05 11:36:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_64(8172898) in pi_balou on r911u28n02 Nov 05 11:36:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172848 in day on r807u03n02 Nov 05 11:36:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172847 in gpu on r808u08n01 Nov 05 11:36:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4018(8172899) in gpu on r4513u26n01 Nov 05 11:36:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4019(8172900) in gpu on r903u34n01 Nov 05 11:36:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172849 NodeList=r815u15n02 #CPUs=1 Partition=day Nov 05 11:36:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172850 NodeList=r815u23n05 #CPUs=1 Partition=day Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8138(8172901) in scavenge on r907u01n04 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8139(8172902) in scavenge on r912u05n04 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8140(8172903) in scavenge on r904u01n01 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8141(8172904) in scavenge on r904u01n01 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8142(8172905) in scavenge on r802u02n01 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8143(8172906) in scavenge on r802u23n02 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8144(8172907) in scavenge on r909u07n04 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8145(8172908) in scavenge on r913u13n03 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8146(8172909) in scavenge on r904u04n01 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8147(8172910) in scavenge on r904u04n02 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8148(8172911) in scavenge on r914u01n01 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8149(8172912) in scavenge on r914u01n03 Nov 05 11:36:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_538(8114805) in scavenge on r806u31n01 Nov 05 11:36:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2957(8172913) in scavenge on r904u01n01 Nov 05 11:36:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2958(8172914) in scavenge on r816u23n08 Nov 05 11:36:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2959(8172915) in scavenge on r806u31n01 Nov 05 11:36:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2960(8172916) in scavenge on r806u31n01 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_27(8172861) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_27(8172861) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_25(8172859) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_25(8172859) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_26(8172860) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_26(8172860) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_54(8172888) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_54(8172888) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_21(8172855) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_21(8172855) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_32(8172866) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_32(8172866) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_34(8172868) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_34(8172868) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_33(8172867) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_33(8172867) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_31(8172865) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_31(8172865) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_35(8172869) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_35(8172869) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_30(8172864) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_30(8172864) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_28(8172862) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_28(8172862) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_29(8172863) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_29(8172863) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_51(8172885) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_51(8172885) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_49(8172883) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_49(8172883) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_52(8172886) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_52(8172886) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_47(8172881) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_47(8172881) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_50(8172884) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_50(8172884) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_53(8172887) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_53(8172887) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_44(8172878) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_44(8172878) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_48(8172882) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_48(8172882) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_45(8172879) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_45(8172879) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_46(8172880) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_46(8172880) done Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_24(8172858) WEXITSTATUS 1 Nov 05 11:36:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_24(8172858) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_39(8172873) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_39(8172873) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_17(8172851) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_17(8172851) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_38(8172872) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_38(8172872) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_37(8172871) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_37(8172871) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_36(8172870) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_36(8172870) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_22(8172856) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_22(8172856) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_23(8172857) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_23(8172857) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_40(8172874) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_40(8172874) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_42(8172876) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_42(8172876) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_18(8172852) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_18(8172852) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_41(8172875) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_41(8172875) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_43(8172877) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_43(8172877) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_57(8172891) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_57(8172891) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_55(8172889) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_55(8172889) done Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_56(8172890) WEXITSTATUS 1 Nov 05 11:37:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_56(8172890) done Nov 05 11:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_19(8172853) WEXITSTATUS 1 Nov 05 11:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_19(8172853) done Nov 05 11:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_20(8172854) WEXITSTATUS 1 Nov 05 11:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_20(8172854) done Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_63(8172897) WEXITSTATUS 1 Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_63(8172897) done Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_62(8172896) WEXITSTATUS 1 Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_62(8172896) done Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_58(8172892) WEXITSTATUS 1 Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_58(8172892) done Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_64(8172898) WEXITSTATUS 1 Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_64(8172898) done Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_59(8172893) WEXITSTATUS 1 Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_59(8172893) done Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_60(8172894) WEXITSTATUS 1 Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_60(8172894) done Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_61(8172895) WEXITSTATUS 1 Nov 05 11:37:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_61(8172895) done Nov 05 11:37:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170656 WEXITSTATUS 0 Nov 05 11:37:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170656 done Nov 05 11:37:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4146(8150162) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:37:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4381(8155311) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:37:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5997(8163002) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:37:19 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6945(8165237) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:37:19 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8169645 had to be killed Nov 05 11:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7914(8170213) WEXITSTATUS 1 Nov 05 11:37:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7914(8170213) done Nov 05 11:37:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4146(8150162) Nov 05 11:37:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4381(8155311) Nov 05 11:37:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6945(8165237) Nov 05 11:37:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5997(8163002) Nov 05 11:37:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172917 InitPrio=31407 usec=2326 Nov 05 11:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7718(8168544) WEXITSTATUS 1 Nov 05 11:37:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7718(8168544) done Nov 05 11:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4018(8172899) WEXITSTATUS 0 Nov 05 11:37:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4018(8172899) done Nov 05 11:37:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:37:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:37:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_144(8114410) WEXITSTATUS 0 Nov 05 11:37:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_144(8114410) done Nov 05 11:37:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_888(7573057) WEXITSTATUS 0 Nov 05 11:37:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_888(7573057) done Nov 05 11:37:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:37:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7859(8169628) WEXITSTATUS 1 Nov 05 11:38:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7859(8169628) done Nov 05 11:38:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:38:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7743(8168846) WEXITSTATUS 1 Nov 05 11:38:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7743(8168846) done Nov 05 11:38:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172595 WEXITSTATUS 0 Nov 05 11:38:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172595 done Nov 05 11:38:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:38:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7886500 WEXITSTATUS 0 Nov 05 11:38:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7886500 done Nov 05 11:38:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:38:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1436(8135969) WEXITSTATUS 0 Nov 05 11:38:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1436(8135969) done Nov 05 11:38:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:38:49 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_30(8114296) Nov 05 11:38:49 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172918 InitPrio=35093 usec=8822 Nov 05 11:38:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:38:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:38:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7817(8169419) WEXITSTATUS 1 Nov 05 11:38:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7817(8169419) done Nov 05 11:38:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:38:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7881(8169913) WEXITSTATUS 1 Nov 05 11:38:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7881(8169913) done Nov 05 11:38:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6618(8164678) WEXITSTATUS 1 Nov 05 11:39:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6618(8164678) done Nov 05 11:39:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_65(8172919) in pi_balou on r802u30n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_66(8172920) in pi_balou on r802u30n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_67(8172921) in pi_balou on r802u30n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_68(8172922) in pi_balou on r802u30n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_69(8172923) in pi_balou on r802u32n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_70(8172924) in pi_balou on r802u32n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_71(8172925) in pi_balou on r802u32n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_72(8172926) in pi_balou on r802u32n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_73(8172927) in pi_balou on r911u09n01 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_74(8172928) in pi_balou on r911u09n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_75(8172929) in pi_balou on r911u09n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_76(8172930) in pi_balou on r911u09n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_77(8172931) in pi_balou on r911u09n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_78(8172932) in pi_balou on r911u09n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_79(8172933) in pi_balou on r911u09n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_80(8172934) in pi_balou on r911u13n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_81(8172935) in pi_balou on r911u13n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_82(8172936) in pi_balou on r911u13n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_83(8172937) in pi_balou on r911u13n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_84(8172938) in pi_balou on r911u13n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_85(8172939) in pi_balou on r911u13n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_86(8172940) in pi_balou on r911u13n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_87(8172941) in pi_balou on r911u13n02 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_88(8172942) in pi_balou on r911u13n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_89(8172943) in pi_balou on r911u13n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_90(8172944) in pi_balou on r911u13n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_91(8172945) in pi_balou on r911u13n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_92(8172946) in pi_balou on r911u13n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_93(8172947) in pi_balou on r911u13n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_94(8172948) in pi_balou on r911u13n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_95(8172949) in pi_balou on r911u13n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_96(8172950) in pi_balou on r911u13n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_97(8172951) in pi_balou on r911u13n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_98(8172952) in pi_balou on r911u13n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_99(8172953) in pi_balou on r911u13n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172813_100(8172813) in pi_balou on r911u13n04 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172917 in day on r918u36n03 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4020(8172954) in gpu on r4513u26n01 Nov 05 11:39:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u03n02 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8150(8172955) in scavenge on r915u32n01 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8151(8172956) in scavenge on r911u24n02 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8152(8172957) in scavenge on r911u24n03 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8153(8172958) in scavenge on r911u28n01 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8154(8172959) in scavenge on r909u04n04 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8155(8172960) in scavenge on r904u10n03 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8156(8172961) in scavenge on r914u05n01 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8157(8172962) in scavenge on r914u05n04 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8158(8172963) in scavenge on r918u05n04 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8159(8172964) in scavenge on r918u32n03 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8160(8172965) in scavenge on r919u01n03 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1040(8115309) in scavenge on r806u23n01 Nov 05 11:39:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 11:39:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2961(8172966) in scavenge on r815u15n09 Nov 05 11:39:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_97(8172951) WEXITSTATUS 1 Nov 05 11:39:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_97(8172951) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_99(8172953) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_99(8172953) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_79(8172933) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_79(8172933) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_96(8172950) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_96(8172950) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_78(8172932) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_78(8172932) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_98(8172952) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_98(8172952) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_77(8172931) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_77(8172931) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_100(8172813) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_100(8172813) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_73(8172927) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_73(8172927) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_88(8172942) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_88(8172942) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_91(8172945) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_91(8172945) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_86(8172940) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_86(8172940) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_87(8172941) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_87(8172941) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_89(8172943) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_89(8172943) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_90(8172944) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_90(8172944) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_85(8172939) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_85(8172939) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_84(8172938) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_84(8172938) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_82(8172936) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_82(8172936) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_83(8172937) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_83(8172937) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_80(8172934) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_80(8172934) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_81(8172935) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_81(8172935) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_76(8172930) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_76(8172930) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_74(8172928) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_74(8172928) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_75(8172929) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_75(8172929) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_94(8172948) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_94(8172948) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_65(8172919) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_65(8172919) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_93(8172947) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_93(8172947) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_92(8172946) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_92(8172946) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_95(8172949) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_95(8172949) done Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_66(8172920) WEXITSTATUS 1 Nov 05 11:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_66(8172920) done Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_70(8172924) WEXITSTATUS 1 Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_70(8172924) done Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_69(8172923) WEXITSTATUS 1 Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_69(8172923) done Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_72(8172926) WEXITSTATUS 1 Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_72(8172926) done Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_71(8172925) WEXITSTATUS 1 Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_71(8172925) done Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_68(8172922) WEXITSTATUS 1 Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_68(8172922) done Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_67(8172921) WEXITSTATUS 1 Nov 05 11:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172813_67(8172921) done Nov 05 11:39:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172967 InitPrio=29025 usec=4929 Nov 05 11:39:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172968 InitPrio=29025 usec=4014 Nov 05 11:39:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172969 InitPrio=29025 usec=3655 Nov 05 11:39:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172970 InitPrio=29025 usec=2952 Nov 05 11:39:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172971 InitPrio=29025 usec=2042 Nov 05 11:39:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172972 InitPrio=29025 usec=1931 Nov 05 11:39:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172973 InitPrio=29025 usec=1831 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172974 InitPrio=29025 usec=1934 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172975 InitPrio=29025 usec=2015 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172976 InitPrio=29025 usec=1921 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172977 InitPrio=29025 usec=1810 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172978 InitPrio=29025 usec=1915 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172979 InitPrio=29025 usec=2054 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172980 InitPrio=29025 usec=2149 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172981 InitPrio=29025 usec=2214 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172982 InitPrio=29025 usec=1996 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172983 InitPrio=29025 usec=1922 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172984 InitPrio=29025 usec=1947 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172985 InitPrio=29025 usec=1815 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172986 InitPrio=29025 usec=1781 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172987 InitPrio=29025 usec=1763 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172988 InitPrio=29025 usec=2290 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172989 InitPrio=29025 usec=2167 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172990 InitPrio=29025 usec=1941 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172991 InitPrio=29025 usec=1935 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172992 InitPrio=29025 usec=1910 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172993 InitPrio=29025 usec=1903 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172994 InitPrio=29025 usec=2248 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172995 InitPrio=29025 usec=1875 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172996 InitPrio=29025 usec=1931 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172997 InitPrio=29025 usec=1868 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172998 InitPrio=29025 usec=1969 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8172999 InitPrio=29025 usec=1894 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173000 InitPrio=29025 usec=1825 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173001 InitPrio=29025 usec=1843 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173002 InitPrio=29025 usec=1810 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173003 InitPrio=29025 usec=1794 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173004 InitPrio=29025 usec=1784 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173005 InitPrio=29025 usec=1764 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173006 InitPrio=29025 usec=1867 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173007 InitPrio=29025 usec=1812 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173008 InitPrio=29025 usec=1806 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173009 InitPrio=29025 usec=1833 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173010 InitPrio=29025 usec=1778 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173011 InitPrio=29025 usec=4094 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173012 InitPrio=29025 usec=3453 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173013 InitPrio=29025 usec=2993 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173014 InitPrio=29025 usec=2887 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173015 InitPrio=29025 usec=2281 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173016 InitPrio=29025 usec=1980 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173017 InitPrio=29025 usec=1926 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173018 InitPrio=29025 usec=1866 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173019 InitPrio=29025 usec=1846 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173020 InitPrio=29025 usec=1881 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173021 InitPrio=29025 usec=1866 Nov 05 11:39:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173022 InitPrio=29025 usec=1897 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173023 InitPrio=29025 usec=2085 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173024 InitPrio=29025 usec=2171 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173025 InitPrio=29025 usec=1970 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173026 InitPrio=29025 usec=1811 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173027 InitPrio=29025 usec=2334 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173028 InitPrio=29025 usec=2132 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173029 InitPrio=29025 usec=1862 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173030 InitPrio=29025 usec=1819 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173031 InitPrio=29025 usec=1790 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173032 InitPrio=29025 usec=1820 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173033 InitPrio=29025 usec=1764 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173034 InitPrio=29025 usec=1745 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173035 InitPrio=29025 usec=1747 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173036 InitPrio=29025 usec=1735 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173037 InitPrio=29025 usec=1853 Nov 05 11:39:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173038 InitPrio=29025 usec=1753 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:39:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:39:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7278(8165951) WEXITSTATUS 1 Nov 05 11:39:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7278(8165951) done Nov 05 11:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7695(8168468) WEXITSTATUS 1 Nov 05 11:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7695(8168468) done Nov 05 11:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2392(8165604) WEXITSTATUS 0 Nov 05 11:40:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2392(8165604) done Nov 05 11:40:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8160142 Nov 05 11:40:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173039 InitPrio=61763 usec=3150 Nov 05 11:40:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173039 NodeList=r816u23n10 #CPUs=1 Partition=pi_anticevic Nov 05 11:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160524 WEXITSTATUS 0 Nov 05 11:40:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160524 done Nov 05 11:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8100(8172741) WEXITSTATUS 1 Nov 05 11:40:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8100(8172741) done Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173040 InitPrio=13477 usec=1942 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173041 InitPrio=13477 usec=1977 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_1(unfulfilled) for JobId=8173041_1(8173042) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_1 uid=23340 usec=166 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_2(unfulfilled) for JobId=8173041_2(8173043) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_2 uid=23340 usec=128 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_3(unfulfilled) for JobId=8173041_3(8173044) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_3 uid=23340 usec=122 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_4(unfulfilled) for JobId=8173041_4(8173045) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_4 uid=23340 usec=249 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_5(unfulfilled) for JobId=8173041_5(8173046) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_5 uid=23340 usec=134 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_6(unfulfilled) for JobId=8173041_6(8173047) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_6 uid=23340 usec=128 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_7(unfulfilled) for JobId=8173041_7(8173048) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_7 uid=23340 usec=138 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_8(unfulfilled) for JobId=8173041_8(8173049) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_8 uid=23340 usec=156 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_9(unfulfilled) for JobId=8173041_9(8173050) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_9 uid=23340 usec=135 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_10(unfulfilled) for JobId=8173041_10(8173051) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_10 uid=23340 usec=122 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_11(unfulfilled) for JobId=8173041_11(8173052) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_11 uid=23340 usec=167 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_12(unfulfilled) for JobId=8173041_12(8173053) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_12 uid=23340 usec=123 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_13(unfulfilled) for JobId=8173041_13(8173054) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_13 uid=23340 usec=128 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_14(unfulfilled) for JobId=8173041_14(8173055) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_14 uid=23340 usec=156 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_15(unfulfilled) for JobId=8173041_15(8173056) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_15 uid=23340 usec=226 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_16(unfulfilled) for JobId=8173041_16(8173057) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_16 uid=23340 usec=146 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_17(unfulfilled) for JobId=8173041_17(8173058) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_17 uid=23340 usec=186 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_18(unfulfilled) for JobId=8173041_18(8173059) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_18 uid=23340 usec=155 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_19(unfulfilled) for JobId=8173041_19(8173060) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_19 uid=23340 usec=162 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_20(unfulfilled) for JobId=8173041_20(8173061) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_20 uid=23340 usec=115 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_21(unfulfilled) for JobId=8173041_21(8173062) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_21 uid=23340 usec=137 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_22(unfulfilled) for JobId=8173041_22(8173063) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_22 uid=23340 usec=126 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_23(unfulfilled) for JobId=8173041_23(8173064) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_23 uid=23340 usec=124 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_24(unfulfilled) for JobId=8173041_24(8173065) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_24 uid=23340 usec=291 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_25(unfulfilled) for JobId=8173041_25(8173066) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_25 uid=23340 usec=155 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_26(unfulfilled) for JobId=8173041_26(8173067) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_26 uid=23340 usec=166 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_27(unfulfilled) for JobId=8173041_27(8173068) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_27 uid=23340 usec=148 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_28(unfulfilled) for JobId=8173041_28(8173069) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_28 uid=23340 usec=157 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_29(unfulfilled) for JobId=8173041_29(8173070) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_29 uid=23340 usec=283 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_30(unfulfilled) for JobId=8173041_30(8173071) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_30 uid=23340 usec=160 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_31(unfulfilled) for JobId=8173041_31(8173072) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_31 uid=23340 usec=160 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_32(unfulfilled) for JobId=8173041_32(8173073) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_32 uid=23340 usec=174 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_33(unfulfilled) for JobId=8173041_33(8173074) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_33 uid=23340 usec=128 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_34(unfulfilled) for JobId=8173041_34(8173075) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_34 uid=23340 usec=218 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_35(unfulfilled) for JobId=8173041_35(8173076) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_35 uid=23340 usec=46333 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_36(unfulfilled) for JobId=8173041_36(8173077) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_36 uid=23340 usec=173 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_37(unfulfilled) for JobId=8173041_37(8173078) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_37 uid=23340 usec=173 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_38(unfulfilled) for JobId=8173041_38(8173079) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_38 uid=23340 usec=163 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_39(unfulfilled) for JobId=8173041_39(8173080) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_39 uid=23340 usec=134 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_40(unfulfilled) for JobId=8173041_40(8173081) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_40 uid=23340 usec=154 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_41(unfulfilled) for JobId=8173041_41(8173082) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_41 uid=23340 usec=144 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_42(unfulfilled) for JobId=8173041_42(8173083) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_42 uid=23340 usec=171 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_43(unfulfilled) for JobId=8173041_43(8173084) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_43 uid=23340 usec=158 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_44(unfulfilled) for JobId=8173041_44(8173085) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_44 uid=23340 usec=164 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_45(unfulfilled) for JobId=8173041_45(8173086) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_45 uid=23340 usec=132 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_46(unfulfilled) for JobId=8173041_46(8173087) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_46 uid=23340 usec=141 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_47(unfulfilled) for JobId=8173041_47(8173088) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_47 uid=23340 usec=156 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_48(unfulfilled) for JobId=8173041_48(8173089) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_48 uid=23340 usec=128 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_49(unfulfilled) for JobId=8173041_49(8173090) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_49 uid=23340 usec=144 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_50(unfulfilled) for JobId=8173041_50(8173091) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_50 uid=23340 usec=138 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_51(unfulfilled) for JobId=8173041_51(8173092) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_51 uid=23340 usec=158 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_52(unfulfilled) for JobId=8173041_52(8173093) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_52 uid=23340 usec=122 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_53(unfulfilled) for JobId=8173041_53(8173094) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_53 uid=23340 usec=152 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_54(unfulfilled) for JobId=8173041_54(8173095) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_54 uid=23340 usec=158 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_55(unfulfilled) for JobId=8173041_55(8173096) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_55 uid=23340 usec=212 Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_56(unfulfilled) for JobId=8173041_56(8173097) Nov 05 11:40:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_56 uid=23340 usec=132 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_57(unfulfilled) for JobId=8173041_57(8173098) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_57 uid=23340 usec=38879 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_58(unfulfilled) for JobId=8173041_58(8173099) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_58 uid=23340 usec=144 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_59(unfulfilled) for JobId=8173041_59(8173100) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_59 uid=23340 usec=146 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_60(unfulfilled) for JobId=8173041_60(8173101) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_60 uid=23340 usec=139 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_61(unfulfilled) for JobId=8173041_61(8173102) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_61 uid=23340 usec=251 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_62(unfulfilled) for JobId=8173041_62(8173103) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_62 uid=23340 usec=136 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_63(unfulfilled) for JobId=8173041_63(8173104) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_63 uid=23340 usec=152 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_64(unfulfilled) for JobId=8173041_64(8173105) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_64 uid=23340 usec=289 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_65(unfulfilled) for JobId=8173041_65(8173106) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_65 uid=23340 usec=124 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_66(unfulfilled) for JobId=8173041_66(8173107) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_66 uid=23340 usec=141 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_67(unfulfilled) for JobId=8173041_67(8173108) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_67 uid=23340 usec=162 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_68(unfulfilled) for JobId=8173041_68(8173109) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_68 uid=23340 usec=125 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_69(unfulfilled) for JobId=8173041_69(8173110) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_69 uid=23340 usec=137 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_70(unfulfilled) for JobId=8173041_70(8173111) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_70 uid=23340 usec=133 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_71(unfulfilled) for JobId=8173041_71(8173112) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_71 uid=23340 usec=186 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_72(unfulfilled) for JobId=8173041_72(8173113) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_72 uid=23340 usec=114 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_73(unfulfilled) for JobId=8173041_73(8173114) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_73 uid=23340 usec=159 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_74(unfulfilled) for JobId=8173041_74(8173115) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_74 uid=23340 usec=161 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_75(unfulfilled) for JobId=8173041_75(8173116) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_75 uid=23340 usec=120 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_76(unfulfilled) for JobId=8173041_76(8173117) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_76 uid=23340 usec=124 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_77(unfulfilled) for JobId=8173041_77(8173118) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_77 uid=23340 usec=152 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_78(unfulfilled) for JobId=8173041_78(8173119) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_78 uid=23340 usec=123 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_79(unfulfilled) for JobId=8173041_79(8173120) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_79 uid=23340 usec=135 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_80(unfulfilled) for JobId=8173041_80(8173121) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_80 uid=23340 usec=131 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_81(unfulfilled) for JobId=8173041_81(8173122) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_81 uid=23340 usec=163 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_82(unfulfilled) for JobId=8173041_82(8173123) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_82 uid=23340 usec=128 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_83(unfulfilled) for JobId=8173041_83(8173124) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_83 uid=23340 usec=157 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_84(unfulfilled) for JobId=8173041_84(8173125) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_84 uid=23340 usec=43859 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_85(unfulfilled) for JobId=8173041_85(8173126) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_85 uid=23340 usec=149 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_86(unfulfilled) for JobId=8173041_86(8173127) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_86 uid=23340 usec=153 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_87(unfulfilled) for JobId=8173041_87(8173128) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_87 uid=23340 usec=153 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_88(unfulfilled) for JobId=8173041_88(8173129) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_88 uid=23340 usec=2176 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_89(unfulfilled) for JobId=8173041_89(8173130) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_89 uid=23340 usec=149 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_90(unfulfilled) for JobId=8173041_90(8173131) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_90 uid=23340 usec=162 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_91(unfulfilled) for JobId=8173041_91(8173132) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_91 uid=23340 usec=172 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_92(unfulfilled) for JobId=8173041_92(8173133) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_92 uid=23340 usec=137 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_93(unfulfilled) for JobId=8173041_93(8173134) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_93 uid=23340 usec=140 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_94(unfulfilled) for JobId=8173041_94(8173135) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_94 uid=23340 usec=154 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_95(unfulfilled) for JobId=8173041_95(8173136) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_95 uid=23340 usec=127 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173040_96(unfulfilled) for JobId=8173041_96(8173041) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173041_96 uid=23340 usec=129 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173137 InitPrio=13477 usec=2405 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_1(unfulfilled) for JobId=8173137_1(8173138) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_1 uid=23340 usec=178 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_2(unfulfilled) for JobId=8173137_2(8173139) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_2 uid=23340 usec=133 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_3(unfulfilled) for JobId=8173137_3(8173140) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_3 uid=23340 usec=155 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_4(unfulfilled) for JobId=8173137_4(8173141) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_4 uid=23340 usec=169 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_5(unfulfilled) for JobId=8173137_5(8173142) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_5 uid=23340 usec=313 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_6(unfulfilled) for JobId=8173137_6(8173143) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_6 uid=23340 usec=134 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_7(unfulfilled) for JobId=8173137_7(8173144) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_7 uid=23340 usec=158 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_8(unfulfilled) for JobId=8173137_8(8173145) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_8 uid=23340 usec=161 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_9(unfulfilled) for JobId=8173137_9(8173146) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_9 uid=23340 usec=228 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_10(unfulfilled) for JobId=8173137_10(8173147) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_10 uid=23340 usec=159 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_11(unfulfilled) for JobId=8173137_11(8173148) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_11 uid=23340 usec=157 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_12(unfulfilled) for JobId=8173137_12(8173149) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_12 uid=23340 usec=124 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_13(unfulfilled) for JobId=8173137_13(8173150) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_13 uid=23340 usec=135 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_14(unfulfilled) for JobId=8173137_14(8173151) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_14 uid=23340 usec=151 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_15(unfulfilled) for JobId=8173137_15(8173152) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_15 uid=23340 usec=128 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_16(unfulfilled) for JobId=8173137_16(8173153) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_16 uid=23340 usec=140 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_17(unfulfilled) for JobId=8173137_17(8173154) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_17 uid=23340 usec=135 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_18(unfulfilled) for JobId=8173137_18(8173155) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_18 uid=23340 usec=172 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_19(unfulfilled) for JobId=8173137_19(8173156) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_19 uid=23340 usec=138 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_20(unfulfilled) for JobId=8173137_20(8173157) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_20 uid=23340 usec=141 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_21(unfulfilled) for JobId=8173137_21(8173158) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_21 uid=23340 usec=166 Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_22(unfulfilled) for JobId=8173137_22(8173159) Nov 05 11:40:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_22 uid=23340 usec=134 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_23(unfulfilled) for JobId=8173137_23(8173160) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_23 uid=23340 usec=24347 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_24(unfulfilled) for JobId=8173137_24(8173161) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_24 uid=23340 usec=199 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_25(unfulfilled) for JobId=8173137_25(8173162) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_25 uid=23340 usec=181 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_26(unfulfilled) for JobId=8173137_26(8173163) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_26 uid=23340 usec=122 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_27(unfulfilled) for JobId=8173137_27(8173164) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_27 uid=23340 usec=141 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_28(unfulfilled) for JobId=8173137_28(8173165) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_28 uid=23340 usec=177 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_29(unfulfilled) for JobId=8173137_29(8173166) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_29 uid=23340 usec=128 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_30(unfulfilled) for JobId=8173137_30(8173167) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_30 uid=23340 usec=142 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_31(unfulfilled) for JobId=8173137_31(8173168) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_31 uid=23340 usec=137 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_32(unfulfilled) for JobId=8173137_32(8173169) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_32 uid=23340 usec=172 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_33(unfulfilled) for JobId=8173137_33(8173170) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_33 uid=23340 usec=125 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_34(unfulfilled) for JobId=8173137_34(8173171) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_34 uid=23340 usec=37672 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_35(unfulfilled) for JobId=8173137_35(8173172) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_35 uid=23340 usec=301 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_36(unfulfilled) for JobId=8173137_36(8173173) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_36 uid=23340 usec=136 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_37(unfulfilled) for JobId=8173137_37(8173174) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_37 uid=23340 usec=135 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_38(unfulfilled) for JobId=8173137_38(8173175) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_38 uid=23340 usec=150 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_39(unfulfilled) for JobId=8173137_39(8173176) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_39 uid=23340 usec=162 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_40(unfulfilled) for JobId=8173137_40(8173177) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_40 uid=23340 usec=122 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_41(unfulfilled) for JobId=8173137_41(8173178) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_41 uid=23340 usec=170 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_42(unfulfilled) for JobId=8173137_42(8173179) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_42 uid=23340 usec=166 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_43(unfulfilled) for JobId=8173137_43(8173180) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_43 uid=23340 usec=124 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_44(unfulfilled) for JobId=8173137_44(8173181) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_44 uid=23340 usec=147 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_45(unfulfilled) for JobId=8173137_45(8173182) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_45 uid=23340 usec=152 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_46(unfulfilled) for JobId=8173137_46(8173183) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_46 uid=23340 usec=126 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_47(unfulfilled) for JobId=8173137_47(8173184) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_47 uid=23340 usec=136 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_48(unfulfilled) for JobId=8173137_48(8173185) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_48 uid=23340 usec=123 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_49(unfulfilled) for JobId=8173137_49(8173186) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_49 uid=23340 usec=150 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_50(unfulfilled) for JobId=8173137_50(8173187) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_50 uid=23340 usec=122 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_51(unfulfilled) for JobId=8173137_51(8173188) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_51 uid=23340 usec=140 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_52(unfulfilled) for JobId=8173137_52(8173189) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_52 uid=23340 usec=159 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_53(unfulfilled) for JobId=8173137_53(8173190) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_53 uid=23340 usec=129 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_54(unfulfilled) for JobId=8173137_54(8173191) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_54 uid=23340 usec=131 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_55(unfulfilled) for JobId=8173137_55(8173192) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_55 uid=23340 usec=153 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_56(unfulfilled) for JobId=8173137_56(8173193) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_56 uid=23340 usec=129 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_57(unfulfilled) for JobId=8173137_57(8173194) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_57 uid=23340 usec=148 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_58(unfulfilled) for JobId=8173137_58(8173195) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_58 uid=23340 usec=125 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_59(unfulfilled) for JobId=8173137_59(8173196) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_59 uid=23340 usec=175 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_60(unfulfilled) for JobId=8173137_60(8173197) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_60 uid=23340 usec=127 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_61(unfulfilled) for JobId=8173137_61(8173198) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_61 uid=23340 usec=140 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_62(unfulfilled) for JobId=8173137_62(8173199) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_62 uid=23340 usec=166 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_63(unfulfilled) for JobId=8173137_63(8173200) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_63 uid=23340 usec=129 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_64(unfulfilled) for JobId=8173137_64(8173201) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_64 uid=23340 usec=128 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_65(unfulfilled) for JobId=8173137_65(8173202) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_65 uid=23340 usec=146 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_66(unfulfilled) for JobId=8173137_66(8173203) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_66 uid=23340 usec=125 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_67(unfulfilled) for JobId=8173137_67(8173204) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_67 uid=23340 usec=138 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_68(unfulfilled) for JobId=8173137_68(8173205) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_68 uid=23340 usec=128 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_69(unfulfilled) for JobId=8173137_69(8173206) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_69 uid=23340 usec=150 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_70(unfulfilled) for JobId=8173137_70(8173207) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_70 uid=23340 usec=128 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_71(unfulfilled) for JobId=8173137_71(8173208) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_71 uid=23340 usec=145 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_72(unfulfilled) for JobId=8173137_72(8173209) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_72 uid=23340 usec=155 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_73(unfulfilled) for JobId=8173137_73(8173210) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_73 uid=23340 usec=123 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_74(unfulfilled) for JobId=8173137_74(8173211) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_74 uid=23340 usec=122 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_75(unfulfilled) for JobId=8173137_75(8173212) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_75 uid=23340 usec=152 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_76(unfulfilled) for JobId=8173137_76(8173213) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_76 uid=23340 usec=121 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_77(unfulfilled) for JobId=8173137_77(8173214) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_77 uid=23340 usec=133 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_78(unfulfilled) for JobId=8173137_78(8173215) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_78 uid=23340 usec=123 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_79(unfulfilled) for JobId=8173137_79(8173216) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_79 uid=23340 usec=150 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_80(unfulfilled) for JobId=8173137_80(8173217) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_80 uid=23340 usec=126 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_81(unfulfilled) for JobId=8173137_81(8173218) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_81 uid=23340 usec=148 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_82(unfulfilled) for JobId=8173137_82(8173219) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_82 uid=23340 usec=156 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_83(unfulfilled) for JobId=8173137_83(8173220) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_83 uid=23340 usec=129 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_84(unfulfilled) for JobId=8173137_84(8173221) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_84 uid=23340 usec=132 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_85(unfulfilled) for JobId=8173137_85(8173222) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_85 uid=23340 usec=40069 Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_86(unfulfilled) for JobId=8173137_86(8173223) Nov 05 11:40:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_86 uid=23340 usec=135 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_87(unfulfilled) for JobId=8173137_87(8173224) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_87 uid=23340 usec=87857 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_88(unfulfilled) for JobId=8173137_88(8173225) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_88 uid=23340 usec=133 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_89(unfulfilled) for JobId=8173137_89(8173226) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_89 uid=23340 usec=229 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_90(unfulfilled) for JobId=8173137_90(8173227) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_90 uid=23340 usec=149 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_91(unfulfilled) for JobId=8173137_91(8173228) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_91 uid=23340 usec=150 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_92(unfulfilled) for JobId=8173137_92(8173229) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_92 uid=23340 usec=158 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_93(unfulfilled) for JobId=8173137_93(8173230) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_93 uid=23340 usec=122 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_94(unfulfilled) for JobId=8173137_94(8173231) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_94 uid=23340 usec=139 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_95(unfulfilled) for JobId=8173137_95(8173232) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_95 uid=23340 usec=124 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173041_96(unfulfilled) for JobId=8173137_96(8173137) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173137_96 uid=23340 usec=141 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173233 InitPrio=13477 usec=1622 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_1(unfulfilled) for JobId=8173233_1(8173234) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_1 uid=23340 usec=254 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_2(unfulfilled) for JobId=8173233_2(8173235) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_2 uid=23340 usec=329 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_3(unfulfilled) for JobId=8173233_3(8173236) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_3 uid=23340 usec=135 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_4(unfulfilled) for JobId=8173233_4(8173237) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_4 uid=23340 usec=141 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_5(unfulfilled) for JobId=8173233_5(8173238) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_5 uid=23340 usec=154 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_6(unfulfilled) for JobId=8173233_6(8173239) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_6 uid=23340 usec=126 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_7(unfulfilled) for JobId=8173233_7(8173240) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_7 uid=23340 usec=140 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_8(unfulfilled) for JobId=8173233_8(8173241) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_8 uid=23340 usec=133 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_9(unfulfilled) for JobId=8173233_9(8173242) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_9 uid=23340 usec=152 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_10(unfulfilled) for JobId=8173233_10(8173243) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_10 uid=23340 usec=115 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_11(unfulfilled) for JobId=8173233_11(8173244) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_11 uid=23340 usec=141 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_12(unfulfilled) for JobId=8173233_12(8173245) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_12 uid=23340 usec=160 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_13(unfulfilled) for JobId=8173233_13(8173246) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_13 uid=23340 usec=123 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_14(unfulfilled) for JobId=8173233_14(8173247) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_14 uid=23340 usec=153 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_15(unfulfilled) for JobId=8173233_15(8173248) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_15 uid=23340 usec=129 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_16(unfulfilled) for JobId=8173233_16(8173249) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_16 uid=23340 usec=160 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_17(unfulfilled) for JobId=8173233_17(8173250) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_17 uid=23340 usec=116 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_18(unfulfilled) for JobId=8173233_18(8173251) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_18 uid=23340 usec=158 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_19(unfulfilled) for JobId=8173233_19(8173252) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_19 uid=23340 usec=169 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_20(unfulfilled) for JobId=8173233_20(8173253) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_20 uid=23340 usec=121 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_21(unfulfilled) for JobId=8173233_21(8173254) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_21 uid=23340 usec=123 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_22(unfulfilled) for JobId=8173233_22(8173255) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_22 uid=23340 usec=136 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_23(unfulfilled) for JobId=8173233_23(8173256) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_23 uid=23340 usec=151 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_24(unfulfilled) for JobId=8173233_24(8173257) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_24 uid=23340 usec=119 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_25(unfulfilled) for JobId=8173233_25(8173258) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_25 uid=23340 usec=159 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_26(unfulfilled) for JobId=8173233_26(8173259) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_26 uid=23340 usec=156 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_27(unfulfilled) for JobId=8173233_27(8173260) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_27 uid=23340 usec=129 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_28(unfulfilled) for JobId=8173233_28(8173261) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_28 uid=23340 usec=138 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_29(unfulfilled) for JobId=8173233_29(8173262) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_29 uid=23340 usec=305 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_30(unfulfilled) for JobId=8173233_30(8173263) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_30 uid=23340 usec=155 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_31(unfulfilled) for JobId=8173233_31(8173264) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_31 uid=23340 usec=113 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_32(unfulfilled) for JobId=8173233_32(8173265) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_32 uid=23340 usec=38138 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_33(unfulfilled) for JobId=8173233_33(8173266) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_33 uid=23340 usec=321 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_34(unfulfilled) for JobId=8173233_34(8173267) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_34 uid=23340 usec=155 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_35(unfulfilled) for JobId=8173233_35(8173268) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_35 uid=23340 usec=139 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_36(unfulfilled) for JobId=8173233_36(8173269) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_36 uid=23340 usec=162 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_37(unfulfilled) for JobId=8173233_37(8173270) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_37 uid=23340 usec=128 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_38(unfulfilled) for JobId=8173233_38(8173271) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_38 uid=23340 usec=143 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_39(unfulfilled) for JobId=8173233_39(8173272) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_39 uid=23340 usec=125 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_40(unfulfilled) for JobId=8173233_40(8173273) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_40 uid=23340 usec=153 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_41(unfulfilled) for JobId=8173233_41(8173274) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_41 uid=23340 usec=115 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_42(unfulfilled) for JobId=8173233_42(8173275) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_42 uid=23340 usec=135 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_43(unfulfilled) for JobId=8173233_43(8173276) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_43 uid=23340 usec=158 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_44(unfulfilled) for JobId=8173233_44(8173277) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_44 uid=23340 usec=137 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_45(unfulfilled) for JobId=8173233_45(8173278) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_45 uid=23340 usec=149 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_46(unfulfilled) for JobId=8173233_46(8173279) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_46 uid=23340 usec=123 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_47(unfulfilled) for JobId=8173233_47(8173280) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_47 uid=23340 usec=135 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_48(unfulfilled) for JobId=8173233_48(8173281) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_48 uid=23340 usec=123 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_49(unfulfilled) for JobId=8173233_49(8173282) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_49 uid=23340 usec=151 Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_50(unfulfilled) for JobId=8173233_50(8173283) Nov 05 11:40:53 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_50 uid=23340 usec=117 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_51(unfulfilled) for JobId=8173233_51(8173284) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_51 uid=23340 usec=104539 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_52(unfulfilled) for JobId=8173233_52(8173285) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_52 uid=23340 usec=276 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_53(unfulfilled) for JobId=8173233_53(8173286) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_53 uid=23340 usec=236 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_54(unfulfilled) for JobId=8173233_54(8173287) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_54 uid=23340 usec=141 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_55(unfulfilled) for JobId=8173233_55(8173288) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_55 uid=23340 usec=156 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_56(unfulfilled) for JobId=8173233_56(8173289) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_56 uid=23340 usec=148 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_57(unfulfilled) for JobId=8173233_57(8173290) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_57 uid=23340 usec=205 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_58(unfulfilled) for JobId=8173233_58(8173291) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_58 uid=23340 usec=139 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_59(unfulfilled) for JobId=8173233_59(8173292) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_59 uid=23340 usec=146 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_60(unfulfilled) for JobId=8173233_60(8173293) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_60 uid=23340 usec=184 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_61(unfulfilled) for JobId=8173233_61(8173294) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_61 uid=23340 usec=159 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_62(unfulfilled) for JobId=8173233_62(8173295) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_62 uid=23340 usec=123 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_63(unfulfilled) for JobId=8173233_63(8173296) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_63 uid=23340 usec=126 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_64(unfulfilled) for JobId=8173233_64(8173297) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_64 uid=23340 usec=317 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_65(unfulfilled) for JobId=8173233_65(8173298) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_65 uid=23340 usec=162 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_66(unfulfilled) for JobId=8173233_66(8173299) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_66 uid=23340 usec=138 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_67(unfulfilled) for JobId=8173233_67(8173300) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_67 uid=23340 usec=150 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_68(unfulfilled) for JobId=8173233_68(8173301) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_68 uid=23340 usec=140 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_69(unfulfilled) for JobId=8173233_69(8173302) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_69 uid=23340 usec=160 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_70(unfulfilled) for JobId=8173233_70(8173303) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_70 uid=23340 usec=124 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_71(unfulfilled) for JobId=8173233_71(8173304) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_71 uid=23340 usec=260 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_72(unfulfilled) for JobId=8173233_72(8173305) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_72 uid=23340 usec=189 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_73(unfulfilled) for JobId=8173233_73(8173306) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_73 uid=23340 usec=179 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_74(unfulfilled) for JobId=8173233_74(8173307) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_74 uid=23340 usec=48151 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_75(unfulfilled) for JobId=8173233_75(8173308) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_75 uid=23340 usec=198 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_76(unfulfilled) for JobId=8173233_76(8173309) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_76 uid=23340 usec=158 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_77(unfulfilled) for JobId=8173233_77(8173310) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_77 uid=23340 usec=184 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_78(unfulfilled) for JobId=8173233_78(8173311) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_78 uid=23340 usec=151 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_79(unfulfilled) for JobId=8173233_79(8173312) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_79 uid=23340 usec=167 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_80(unfulfilled) for JobId=8173233_80(8173313) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_80 uid=23340 usec=146 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_81(unfulfilled) for JobId=8173233_81(8173314) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_81 uid=23340 usec=183 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_82(unfulfilled) for JobId=8173233_82(8173315) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_82 uid=23340 usec=164 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_83(unfulfilled) for JobId=8173233_83(8173316) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_83 uid=23340 usec=173 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_84(unfulfilled) for JobId=8173233_84(8173317) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_84 uid=23340 usec=142 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_85(unfulfilled) for JobId=8173233_85(8173318) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_85 uid=23340 usec=153 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_86(unfulfilled) for JobId=8173233_86(8173319) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_86 uid=23340 usec=162 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_87(unfulfilled) for JobId=8173233_87(8173320) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_87 uid=23340 usec=177 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_88(unfulfilled) for JobId=8173233_88(8173321) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_88 uid=23340 usec=146 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_89(unfulfilled) for JobId=8173233_89(8173322) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_89 uid=23340 usec=147 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_90(unfulfilled) for JobId=8173233_90(8173323) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_90 uid=23340 usec=175 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_91(unfulfilled) for JobId=8173233_91(8173324) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_91 uid=23340 usec=142 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_92(unfulfilled) for JobId=8173233_92(8173325) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_92 uid=23340 usec=155 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_93(unfulfilled) for JobId=8173233_93(8173326) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_93 uid=23340 usec=209 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_94(unfulfilled) for JobId=8173233_94(8173327) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_94 uid=23340 usec=182 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_95(unfulfilled) for JobId=8173233_95(8173328) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_95 uid=23340 usec=134 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173137_96(unfulfilled) for JobId=8173233_96(8173233) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173233_96 uid=23340 usec=146 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173329 InitPrio=13477 usec=2461 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_1(unfulfilled) for JobId=8173329_1(8173330) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_1 uid=23340 usec=149 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_2(unfulfilled) for JobId=8173329_2(8173331) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_2 uid=23340 usec=191 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_3(unfulfilled) for JobId=8173329_3(8173332) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_3 uid=23340 usec=143 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_4(unfulfilled) for JobId=8173329_4(8173333) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_4 uid=23340 usec=157 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_5(unfulfilled) for JobId=8173329_5(8173334) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_5 uid=23340 usec=139 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_6(unfulfilled) for JobId=8173329_6(8173335) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_6 uid=23340 usec=198 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_7(unfulfilled) for JobId=8173329_7(8173336) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_7 uid=23340 usec=187 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_8(unfulfilled) for JobId=8173329_8(8173337) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_8 uid=23340 usec=146 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_9(unfulfilled) for JobId=8173329_9(8173338) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_9 uid=23340 usec=172 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_10(unfulfilled) for JobId=8173329_10(8173339) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_10 uid=23340 usec=173 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_11(unfulfilled) for JobId=8173329_11(8173340) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_11 uid=23340 usec=137 Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_12(unfulfilled) for JobId=8173329_12(8173341) Nov 05 11:40:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_12 uid=23340 usec=141 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_13(unfulfilled) for JobId=8173329_13(8173342) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_13 uid=23340 usec=19764 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_14(unfulfilled) for JobId=8173329_14(8173343) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_14 uid=23340 usec=171 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_15(unfulfilled) for JobId=8173329_15(8173344) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_15 uid=23340 usec=144 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_16(unfulfilled) for JobId=8173329_16(8173345) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_16 uid=23340 usec=127 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_17(unfulfilled) for JobId=8173329_17(8173346) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_17 uid=23340 usec=155 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_18(unfulfilled) for JobId=8173329_18(8173347) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_18 uid=23340 usec=119 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_19(unfulfilled) for JobId=8173329_19(8173348) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_19 uid=23340 usec=143 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_20(unfulfilled) for JobId=8173329_20(8173349) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_20 uid=23340 usec=207 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_21(unfulfilled) for JobId=8173329_21(8173350) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_21 uid=23340 usec=296 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_22(unfulfilled) for JobId=8173329_22(8173351) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_22 uid=23340 usec=127 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_23(unfulfilled) for JobId=8173329_23(8173352) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_23 uid=23340 usec=151 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_24(unfulfilled) for JobId=8173329_24(8173353) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_24 uid=23340 usec=177 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_25(unfulfilled) for JobId=8173329_25(8173354) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_25 uid=23340 usec=152 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_26(unfulfilled) for JobId=8173329_26(8173355) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_26 uid=23340 usec=35341 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_27(unfulfilled) for JobId=8173329_27(8173356) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_27 uid=23340 usec=247 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_28(unfulfilled) for JobId=8173329_28(8173357) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_28 uid=23340 usec=159 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_29(unfulfilled) for JobId=8173329_29(8173358) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_29 uid=23340 usec=140 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_30(unfulfilled) for JobId=8173329_30(8173359) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_30 uid=23340 usec=158 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_31(unfulfilled) for JobId=8173329_31(8173360) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_31 uid=23340 usec=125 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_32(unfulfilled) for JobId=8173329_32(8173361) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_32 uid=23340 usec=140 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_33(unfulfilled) for JobId=8173329_33(8173362) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_33 uid=23340 usec=126 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_34(unfulfilled) for JobId=8173329_34(8173363) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_34 uid=23340 usec=178 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_35(unfulfilled) for JobId=8173329_35(8173364) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_35 uid=23340 usec=146 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_36(unfulfilled) for JobId=8173329_36(8173365) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_36 uid=23340 usec=151 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_37(unfulfilled) for JobId=8173329_37(8173366) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_37 uid=23340 usec=189 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_38(unfulfilled) for JobId=8173329_38(8173367) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_38 uid=23340 usec=179 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_39(unfulfilled) for JobId=8173329_39(8173368) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_39 uid=23340 usec=139 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_40(unfulfilled) for JobId=8173329_40(8173369) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_40 uid=23340 usec=137 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_41(unfulfilled) for JobId=8173329_41(8173370) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_41 uid=23340 usec=155 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_42(unfulfilled) for JobId=8173329_42(8173371) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_42 uid=23340 usec=139 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_43(unfulfilled) for JobId=8173329_43(8173372) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_43 uid=23340 usec=179 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_44(unfulfilled) for JobId=8173329_44(8173373) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_44 uid=23340 usec=195 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_45(unfulfilled) for JobId=8173329_45(8173374) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_45 uid=23340 usec=185 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_46(unfulfilled) for JobId=8173329_46(8173375) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_46 uid=23340 usec=127 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_47(unfulfilled) for JobId=8173329_47(8173376) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_47 uid=23340 usec=176 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_48(unfulfilled) for JobId=8173329_48(8173377) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_48 uid=23340 usec=175 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_49(unfulfilled) for JobId=8173329_49(8173378) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_49 uid=23340 usec=152 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_50(unfulfilled) for JobId=8173329_50(8173379) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_50 uid=23340 usec=167 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_51(unfulfilled) for JobId=8173329_51(8173380) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_51 uid=23340 usec=148 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_52(unfulfilled) for JobId=8173329_52(8173381) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_52 uid=23340 usec=350 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_53(unfulfilled) for JobId=8173329_53(8173382) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_53 uid=23340 usec=137 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_54(unfulfilled) for JobId=8173329_54(8173383) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_54 uid=23340 usec=161 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_55(unfulfilled) for JobId=8173329_55(8173384) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_55 uid=23340 usec=178 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_56(unfulfilled) for JobId=8173329_56(8173385) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_56 uid=23340 usec=146 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_57(unfulfilled) for JobId=8173329_57(8173386) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_57 uid=23340 usec=162 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_58(unfulfilled) for JobId=8173329_58(8173387) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_58 uid=23340 usec=137 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_59(unfulfilled) for JobId=8173329_59(8173388) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_59 uid=23340 usec=155 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_60(unfulfilled) for JobId=8173329_60(8173389) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_60 uid=23340 usec=133 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_61(unfulfilled) for JobId=8173329_61(8173390) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_61 uid=23340 usec=169 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_62(unfulfilled) for JobId=8173329_62(8173391) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_62 uid=23340 usec=195 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_63(unfulfilled) for JobId=8173329_63(8173392) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_63 uid=23340 usec=153 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7799(8169342) WEXITSTATUS 1 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7799(8169342) done Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_64(unfulfilled) for JobId=8173329_64(8173393) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_64 uid=23340 usec=182 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_65(unfulfilled) for JobId=8173329_65(8173394) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_65 uid=23340 usec=146 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_66(unfulfilled) for JobId=8173329_66(8173395) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_66 uid=23340 usec=135 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_67(unfulfilled) for JobId=8173329_67(8173396) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_67 uid=23340 usec=165 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_68(unfulfilled) for JobId=8173329_68(8173397) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_68 uid=23340 usec=122 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_69(unfulfilled) for JobId=8173329_69(8173398) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_69 uid=23340 usec=142 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_70(unfulfilled) for JobId=8173329_70(8173399) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_70 uid=23340 usec=161 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_71(unfulfilled) for JobId=8173329_71(8173400) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_71 uid=23340 usec=149 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_72(unfulfilled) for JobId=8173329_72(8173401) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_72 uid=23340 usec=153 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_73(unfulfilled) for JobId=8173329_73(8173402) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_73 uid=23340 usec=143 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_74(unfulfilled) for JobId=8173329_74(8173403) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_74 uid=23340 usec=283 Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_75(unfulfilled) for JobId=8173329_75(8173404) Nov 05 11:40:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_75 uid=23340 usec=193 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_76(unfulfilled) for JobId=8173329_76(8173405) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_76 uid=23340 usec=47531 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_77(unfulfilled) for JobId=8173329_77(8173406) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_77 uid=23340 usec=41386 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_78(unfulfilled) for JobId=8173329_78(8173407) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_78 uid=23340 usec=195 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_79(unfulfilled) for JobId=8173329_79(8173408) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_79 uid=23340 usec=133 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_80(unfulfilled) for JobId=8173329_80(8173409) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_80 uid=23340 usec=141 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_81(unfulfilled) for JobId=8173329_81(8173410) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_81 uid=23340 usec=255 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_82(unfulfilled) for JobId=8173329_82(8173411) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_82 uid=23340 usec=136 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_83(unfulfilled) for JobId=8173329_83(8173412) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_83 uid=23340 usec=161 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_84(unfulfilled) for JobId=8173329_84(8173413) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_84 uid=23340 usec=175 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_85(unfulfilled) for JobId=8173329_85(8173414) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_85 uid=23340 usec=138 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_86(unfulfilled) for JobId=8173329_86(8173415) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_86 uid=23340 usec=138 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_87(unfulfilled) for JobId=8173329_87(8173416) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_87 uid=23340 usec=161 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_88(unfulfilled) for JobId=8173329_88(8173417) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_88 uid=23340 usec=132 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_89(unfulfilled) for JobId=8173329_89(8173418) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_89 uid=23340 usec=146 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_90(unfulfilled) for JobId=8173329_90(8173419) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_90 uid=23340 usec=184 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_91(unfulfilled) for JobId=8173329_91(8173420) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_91 uid=23340 usec=178 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_92(unfulfilled) for JobId=8173329_92(8173421) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_92 uid=23340 usec=144 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_93(unfulfilled) for JobId=8173329_93(8173422) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_93 uid=23340 usec=140 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_94(unfulfilled) for JobId=8173329_94(8173423) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_94 uid=23340 usec=276 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_95(unfulfilled) for JobId=8173329_95(8173424) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_95 uid=23340 usec=155 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173233_96(unfulfilled) for JobId=8173329_96(8173329) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173329_96 uid=23340 usec=142 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173425 InitPrio=13477 usec=3316 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_1(unfulfilled) for JobId=8173425_1(8173426) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_1 uid=23340 usec=183 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_2(unfulfilled) for JobId=8173425_2(8173427) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_2 uid=23340 usec=166 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_3(unfulfilled) for JobId=8173425_3(8173428) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_3 uid=23340 usec=171 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_4(unfulfilled) for JobId=8173425_4(8173429) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_4 uid=23340 usec=176 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_5(unfulfilled) for JobId=8173425_5(8173430) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_5 uid=23340 usec=146 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_6(unfulfilled) for JobId=8173425_6(8173431) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_6 uid=23340 usec=164 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_7(unfulfilled) for JobId=8173425_7(8173432) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_7 uid=23340 usec=142 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_8(unfulfilled) for JobId=8173425_8(8173433) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_8 uid=23340 usec=177 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_9(unfulfilled) for JobId=8173425_9(8173434) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_9 uid=23340 usec=137 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_10(unfulfilled) for JobId=8173425_10(8173435) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_10 uid=23340 usec=175 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_11(unfulfilled) for JobId=8173425_11(8173436) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_11 uid=23340 usec=173 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_12(unfulfilled) for JobId=8173425_12(8173437) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_12 uid=23340 usec=152 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_13(unfulfilled) for JobId=8173425_13(8173438) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_13 uid=23340 usec=208 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_14(unfulfilled) for JobId=8173425_14(8173439) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_14 uid=23340 usec=177 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_15(unfulfilled) for JobId=8173425_15(8173440) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_15 uid=23340 usec=145 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_16(unfulfilled) for JobId=8173425_16(8173441) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_16 uid=23340 usec=156 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_17(unfulfilled) for JobId=8173425_17(8173442) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_17 uid=23340 usec=147 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_18(unfulfilled) for JobId=8173425_18(8173443) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_18 uid=23340 usec=190 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_19(unfulfilled) for JobId=8173425_19(8173444) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_19 uid=23340 usec=159 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_20(unfulfilled) for JobId=8173425_20(8173445) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_20 uid=23340 usec=154 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_21(unfulfilled) for JobId=8173425_21(8173446) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_21 uid=23340 usec=172 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_22(unfulfilled) for JobId=8173425_22(8173447) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_22 uid=23340 usec=174 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_23(unfulfilled) for JobId=8173425_23(8173448) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_23 uid=23340 usec=143 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_24(unfulfilled) for JobId=8173425_24(8173449) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_24 uid=23340 usec=144 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_25(unfulfilled) for JobId=8173425_25(8173450) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_25 uid=23340 usec=179 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_26(unfulfilled) for JobId=8173425_26(8173451) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_26 uid=23340 usec=146 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_27(unfulfilled) for JobId=8173425_27(8173452) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_27 uid=23340 usec=265 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_28(unfulfilled) for JobId=8173425_28(8173453) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_28 uid=23340 usec=42226 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_29(unfulfilled) for JobId=8173425_29(8173454) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_29 uid=23340 usec=268 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_30(unfulfilled) for JobId=8173425_30(8173455) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_30 uid=23340 usec=162 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_31(unfulfilled) for JobId=8173425_31(8173456) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_31 uid=23340 usec=165 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_32(unfulfilled) for JobId=8173425_32(8173457) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_32 uid=23340 usec=173 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_33(unfulfilled) for JobId=8173425_33(8173458) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_33 uid=23340 usec=143 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_34(unfulfilled) for JobId=8173425_34(8173459) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_34 uid=23340 usec=158 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_35(unfulfilled) for JobId=8173425_35(8173460) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_35 uid=23340 usec=141 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_36(unfulfilled) for JobId=8173425_36(8173461) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_36 uid=23340 usec=165 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_37(unfulfilled) for JobId=8173425_37(8173462) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_37 uid=23340 usec=144 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_38(unfulfilled) for JobId=8173425_38(8173463) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_38 uid=23340 usec=158 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_39(unfulfilled) for JobId=8173425_39(8173464) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_39 uid=23340 usec=171 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_40(unfulfilled) for JobId=8173425_40(8173465) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_40 uid=23340 usec=141 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_41(unfulfilled) for JobId=8173425_41(8173466) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_41 uid=23340 usec=154 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_42(unfulfilled) for JobId=8173425_42(8173467) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_42 uid=23340 usec=145 Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_43(unfulfilled) for JobId=8173425_43(8173468) Nov 05 11:40:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_43 uid=23340 usec=176 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_44(unfulfilled) for JobId=8173425_44(8173469) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_44 uid=23340 usec=22068 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_45(unfulfilled) for JobId=8173425_45(8173470) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_45 uid=23340 usec=170 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_46(unfulfilled) for JobId=8173425_46(8173471) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_46 uid=23340 usec=218 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_47(unfulfilled) for JobId=8173425_47(8173472) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_47 uid=23340 usec=148 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_48(unfulfilled) for JobId=8173425_48(8173473) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_48 uid=23340 usec=135 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_49(unfulfilled) for JobId=8173425_49(8173474) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_49 uid=23340 usec=163 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_50(unfulfilled) for JobId=8173425_50(8173475) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_50 uid=23340 usec=131 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_51(unfulfilled) for JobId=8173425_51(8173476) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_51 uid=23340 usec=143 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_52(unfulfilled) for JobId=8173425_52(8173477) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_52 uid=23340 usec=312 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_53(unfulfilled) for JobId=8173425_53(8173478) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_53 uid=23340 usec=133 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_54(unfulfilled) for JobId=8173425_54(8173479) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_54 uid=23340 usec=156 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_55(unfulfilled) for JobId=8173425_55(8173480) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_55 uid=23340 usec=162 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_56(unfulfilled) for JobId=8173425_56(8173481) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_56 uid=23340 usec=136 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_57(unfulfilled) for JobId=8173425_57(8173482) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_57 uid=23340 usec=301 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_58(unfulfilled) for JobId=8173425_58(8173483) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_58 uid=23340 usec=165 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_59(unfulfilled) for JobId=8173425_59(8173484) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_59 uid=23340 usec=131 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_60(unfulfilled) for JobId=8173425_60(8173485) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_60 uid=23340 usec=150 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_61(unfulfilled) for JobId=8173425_61(8173486) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_61 uid=23340 usec=230 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_62(unfulfilled) for JobId=8173425_62(8173487) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_62 uid=23340 usec=192 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_63(unfulfilled) for JobId=8173425_63(8173488) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_63 uid=23340 usec=167 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_64(unfulfilled) for JobId=8173425_64(8173489) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_64 uid=23340 usec=148 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_65(unfulfilled) for JobId=8173425_65(8173490) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_65 uid=23340 usec=127 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_66(unfulfilled) for JobId=8173425_66(8173491) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_66 uid=23340 usec=182 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_67(unfulfilled) for JobId=8173425_67(8173492) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_67 uid=23340 usec=149 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_68(unfulfilled) for JobId=8173425_68(8173493) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_68 uid=23340 usec=178 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_69(unfulfilled) for JobId=8173425_69(8173494) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_69 uid=23340 usec=138 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_70(unfulfilled) for JobId=8173425_70(8173495) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_70 uid=23340 usec=137 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_71(unfulfilled) for JobId=8173425_71(8173496) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_71 uid=23340 usec=158 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_72(unfulfilled) for JobId=8173425_72(8173497) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_72 uid=23340 usec=131 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_73(unfulfilled) for JobId=8173425_73(8173498) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_73 uid=23340 usec=143 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_74(unfulfilled) for JobId=8173425_74(8173499) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_74 uid=23340 usec=135 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_75(unfulfilled) for JobId=8173425_75(8173500) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_75 uid=23340 usec=159 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_76(unfulfilled) for JobId=8173425_76(8173501) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_76 uid=23340 usec=140 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_77(unfulfilled) for JobId=8173425_77(8173502) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_77 uid=23340 usec=138 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_78(unfulfilled) for JobId=8173425_78(8173503) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_78 uid=23340 usec=305 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_79(unfulfilled) for JobId=8173425_79(8173504) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_79 uid=23340 usec=42156 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_80(unfulfilled) for JobId=8173425_80(8173505) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_80 uid=23340 usec=220 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_81(unfulfilled) for JobId=8173425_81(8173506) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_81 uid=23340 usec=166 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_82(unfulfilled) for JobId=8173425_82(8173507) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_82 uid=23340 usec=172 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_83(unfulfilled) for JobId=8173425_83(8173508) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_83 uid=23340 usec=139 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_84(unfulfilled) for JobId=8173425_84(8173509) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_84 uid=23340 usec=127 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_85(unfulfilled) for JobId=8173425_85(8173510) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_85 uid=23340 usec=155 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_86(unfulfilled) for JobId=8173425_86(8173511) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_86 uid=23340 usec=126 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_87(unfulfilled) for JobId=8173425_87(8173512) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_87 uid=23340 usec=137 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_88(unfulfilled) for JobId=8173425_88(8173513) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_88 uid=23340 usec=160 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_89(unfulfilled) for JobId=8173425_89(8173514) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_89 uid=23340 usec=145 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_90(unfulfilled) for JobId=8173425_90(8173515) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_90 uid=23340 usec=773 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_91(unfulfilled) for JobId=8173425_91(8173516) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_91 uid=23340 usec=137 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_92(unfulfilled) for JobId=8173425_92(8173517) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_92 uid=23340 usec=164 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_93(unfulfilled) for JobId=8173425_93(8173518) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_93 uid=23340 usec=128 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_94(unfulfilled) for JobId=8173425_94(8173519) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_94 uid=23340 usec=152 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_95(unfulfilled) for JobId=8173425_95(8173520) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_95 uid=23340 usec=131 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173329_96(unfulfilled) for JobId=8173425_96(8173425) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173425_96 uid=23340 usec=140 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173521 InitPrio=13477 usec=2645 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_1(unfulfilled) for JobId=8173521_1(8173522) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_1 uid=23340 usec=173 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_2(unfulfilled) for JobId=8173521_2(8173523) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_2 uid=23340 usec=172 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_3(unfulfilled) for JobId=8173521_3(8173524) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_3 uid=23340 usec=141 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_4(unfulfilled) for JobId=8173521_4(8173525) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_4 uid=23340 usec=135 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_5(unfulfilled) for JobId=8173521_5(8173526) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_5 uid=23340 usec=160 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_6(unfulfilled) for JobId=8173521_6(8173527) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_6 uid=23340 usec=134 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_7(unfulfilled) for JobId=8173521_7(8173528) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_7 uid=23340 usec=149 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_8(unfulfilled) for JobId=8173521_8(8173529) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_8 uid=23340 usec=129 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_9(unfulfilled) for JobId=8173521_9(8173530) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_9 uid=23340 usec=166 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_10(unfulfilled) for JobId=8173521_10(8173531) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_10 uid=23340 usec=129 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_11(unfulfilled) for JobId=8173521_11(8173532) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_11 uid=23340 usec=143 Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_12(unfulfilled) for JobId=8173521_12(8173533) Nov 05 11:40:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_12 uid=23340 usec=160 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_13(unfulfilled) for JobId=8173521_13(8173534) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_13 uid=23340 usec=15635 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_14(unfulfilled) for JobId=8173521_14(8173535) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_14 uid=23340 usec=1139 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_15(unfulfilled) for JobId=8173521_15(8173536) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_15 uid=23340 usec=3397 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_16(unfulfilled) for JobId=8173521_16(8173537) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_16 uid=23340 usec=724 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_17(unfulfilled) for JobId=8173521_17(8173538) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_17 uid=23340 usec=178 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_18(unfulfilled) for JobId=8173521_18(8173539) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_18 uid=23340 usec=153 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_19(unfulfilled) for JobId=8173521_19(8173540) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_19 uid=23340 usec=177 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_20(unfulfilled) for JobId=8173521_20(8173541) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_20 uid=23340 usec=124 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_21(unfulfilled) for JobId=8173521_21(8173542) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_21 uid=23340 usec=157 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_22(unfulfilled) for JobId=8173521_22(8173543) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_22 uid=23340 usec=154 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_23(unfulfilled) for JobId=8173521_23(8173544) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_23 uid=23340 usec=116 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_24(unfulfilled) for JobId=8173521_24(8173545) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_24 uid=23340 usec=111 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_25(unfulfilled) for JobId=8173521_25(8173546) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_25 uid=23340 usec=154 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_26(unfulfilled) for JobId=8173521_26(8173547) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_26 uid=23340 usec=129 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_27(unfulfilled) for JobId=8173521_27(8173548) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_27 uid=23340 usec=130 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_28(unfulfilled) for JobId=8173521_28(8173549) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_28 uid=23340 usec=131 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_29(unfulfilled) for JobId=8173521_29(8173550) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_29 uid=23340 usec=285 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_30(unfulfilled) for JobId=8173521_30(8173551) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_30 uid=23340 usec=131 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_31(unfulfilled) for JobId=8173521_31(8173552) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_31 uid=23340 usec=128 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_32(unfulfilled) for JobId=8173521_32(8173553) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_32 uid=23340 usec=47028 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_33(unfulfilled) for JobId=8173521_33(8173554) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_33 uid=23340 usec=183 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_34(unfulfilled) for JobId=8173521_34(8173555) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_34 uid=23340 usec=137 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_35(unfulfilled) for JobId=8173521_35(8173556) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_35 uid=23340 usec=135 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_36(unfulfilled) for JobId=8173521_36(8173557) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_36 uid=23340 usec=140 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_37(unfulfilled) for JobId=8173521_37(8173558) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_37 uid=23340 usec=126 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_38(unfulfilled) for JobId=8173521_38(8173559) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_38 uid=23340 usec=125 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_39(unfulfilled) for JobId=8173521_39(8173560) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_39 uid=23340 usec=132 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_40(unfulfilled) for JobId=8173521_40(8173561) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_40 uid=23340 usec=135 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_41(unfulfilled) for JobId=8173521_41(8173562) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_41 uid=23340 usec=129 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_42(unfulfilled) for JobId=8173521_42(8173563) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_42 uid=23340 usec=127 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_43(unfulfilled) for JobId=8173521_43(8173564) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_43 uid=23340 usec=123 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_44(unfulfilled) for JobId=8173521_44(8173565) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_44 uid=23340 usec=122 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_45(unfulfilled) for JobId=8173521_45(8173566) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_45 uid=23340 usec=121 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_46(unfulfilled) for JobId=8173521_46(8173567) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_46 uid=23340 usec=122 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_47(unfulfilled) for JobId=8173521_47(8173568) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_47 uid=23340 usec=135 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_48(unfulfilled) for JobId=8173521_48(8173569) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_48 uid=23340 usec=124 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_49(unfulfilled) for JobId=8173521_49(8173570) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_49 uid=23340 usec=197 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_50(unfulfilled) for JobId=8173521_50(8173571) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_50 uid=23340 usec=170 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_51(unfulfilled) for JobId=8173521_51(8173572) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_51 uid=23340 usec=126 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_52(unfulfilled) for JobId=8173521_52(8173573) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_52 uid=23340 usec=126 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_53(unfulfilled) for JobId=8173521_53(8173574) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_53 uid=23340 usec=126 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_54(unfulfilled) for JobId=8173521_54(8173575) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_54 uid=23340 usec=158 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_55(unfulfilled) for JobId=8173521_55(8173576) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_55 uid=23340 usec=199 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_56(unfulfilled) for JobId=8173521_56(8173577) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_56 uid=23340 usec=156 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_57(unfulfilled) for JobId=8173521_57(8173578) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_57 uid=23340 usec=311 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_58(unfulfilled) for JobId=8173521_58(8173579) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_58 uid=23340 usec=122 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_59(unfulfilled) for JobId=8173521_59(8173580) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_59 uid=23340 usec=173 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_60(unfulfilled) for JobId=8173521_60(8173581) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_60 uid=23340 usec=133 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_61(unfulfilled) for JobId=8173521_61(8173582) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_61 uid=23340 usec=154 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_62(unfulfilled) for JobId=8173521_62(8173583) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_62 uid=23340 usec=151 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_63(unfulfilled) for JobId=8173521_63(8173584) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_63 uid=23340 usec=158 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_64(unfulfilled) for JobId=8173521_64(8173585) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_64 uid=23340 usec=117 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_65(unfulfilled) for JobId=8173521_65(8173586) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_65 uid=23340 usec=128 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_66(unfulfilled) for JobId=8173521_66(8173587) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_66 uid=23340 usec=185 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_67(unfulfilled) for JobId=8173521_67(8173588) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_67 uid=23340 usec=134 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_68(unfulfilled) for JobId=8173521_68(8173589) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_68 uid=23340 usec=155 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_69(unfulfilled) for JobId=8173521_69(8173590) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_69 uid=23340 usec=152 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_70(unfulfilled) for JobId=8173521_70(8173591) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_70 uid=23340 usec=157 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_71(unfulfilled) for JobId=8173521_71(8173592) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_71 uid=23340 usec=127 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_72(unfulfilled) for JobId=8173521_72(8173593) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_72 uid=23340 usec=119 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_73(unfulfilled) for JobId=8173521_73(8173594) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_73 uid=23340 usec=167 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_74(unfulfilled) for JobId=8173521_74(8173595) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_74 uid=23340 usec=134 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_75(unfulfilled) for JobId=8173521_75(8173596) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_75 uid=23340 usec=148 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_76(unfulfilled) for JobId=8173521_76(8173597) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_76 uid=23340 usec=142 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_77(unfulfilled) for JobId=8173521_77(8173598) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_77 uid=23340 usec=139 Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_78(unfulfilled) for JobId=8173521_78(8173599) Nov 05 11:40:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_78 uid=23340 usec=122 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_79(unfulfilled) for JobId=8173521_79(8173600) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_79 uid=23340 usec=141647 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_80(unfulfilled) for JobId=8173521_80(8173601) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_80 uid=23340 usec=30823 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_81(unfulfilled) for JobId=8173521_81(8173602) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_81 uid=23340 usec=133 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_82(unfulfilled) for JobId=8173521_82(8173603) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_82 uid=23340 usec=152 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_83(unfulfilled) for JobId=8173521_83(8173604) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_83 uid=23340 usec=125 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_84(unfulfilled) for JobId=8173521_84(8173605) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_84 uid=23340 usec=122 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_85(unfulfilled) for JobId=8173521_85(8173606) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_85 uid=23340 usec=126 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_86(unfulfilled) for JobId=8173521_86(8173607) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_86 uid=23340 usec=116 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_87(unfulfilled) for JobId=8173521_87(8173608) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_87 uid=23340 usec=122 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_88(unfulfilled) for JobId=8173521_88(8173609) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_88 uid=23340 usec=114 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_89(unfulfilled) for JobId=8173521_89(8173610) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_89 uid=23340 usec=169 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_90(unfulfilled) for JobId=8173521_90(8173611) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_90 uid=23340 usec=143 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_91(unfulfilled) for JobId=8173521_91(8173612) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_91 uid=23340 usec=113 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_92(unfulfilled) for JobId=8173521_92(8173613) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_92 uid=23340 usec=120 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_93(unfulfilled) for JobId=8173521_93(8173614) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_93 uid=23340 usec=112 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_94(unfulfilled) for JobId=8173521_94(8173615) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_94 uid=23340 usec=113 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_95(unfulfilled) for JobId=8173521_95(8173616) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_95 uid=23340 usec=110 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173425_96(unfulfilled) for JobId=8173521_96(8173521) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173521_96 uid=23340 usec=265 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173617 InitPrio=13477 usec=2265 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_1(unfulfilled) for JobId=8173617_1(8173618) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_1 uid=23340 usec=310 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_2(unfulfilled) for JobId=8173617_2(8173619) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_2 uid=23340 usec=132 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_3(unfulfilled) for JobId=8173617_3(8173620) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_3 uid=23340 usec=121 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_4(unfulfilled) for JobId=8173617_4(8173621) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_4 uid=23340 usec=116 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_5(unfulfilled) for JobId=8173617_5(8173622) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_5 uid=23340 usec=135 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_6(unfulfilled) for JobId=8173617_6(8173623) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_6 uid=23340 usec=121 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_7(unfulfilled) for JobId=8173617_7(8173624) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_7 uid=23340 usec=120 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_8(unfulfilled) for JobId=8173617_8(8173625) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_8 uid=23340 usec=120 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_9(unfulfilled) for JobId=8173617_9(8173626) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_9 uid=23340 usec=122 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_10(unfulfilled) for JobId=8173617_10(8173627) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_10 uid=23340 usec=128 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_11(unfulfilled) for JobId=8173617_11(8173628) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_11 uid=23340 usec=127 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_12(unfulfilled) for JobId=8173617_12(8173629) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_12 uid=23340 usec=125 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_13(unfulfilled) for JobId=8173617_13(8173630) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_13 uid=23340 usec=117 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_14(unfulfilled) for JobId=8173617_14(8173631) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_14 uid=23340 usec=138 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_15(unfulfilled) for JobId=8173617_15(8173632) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_15 uid=23340 usec=125 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_16(unfulfilled) for JobId=8173617_16(8173633) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_16 uid=23340 usec=871 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_17(unfulfilled) for JobId=8173617_17(8173634) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_17 uid=23340 usec=146 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_18(unfulfilled) for JobId=8173617_18(8173635) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_18 uid=23340 usec=118 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_19(unfulfilled) for JobId=8173617_19(8173636) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_19 uid=23340 usec=123 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_20(unfulfilled) for JobId=8173617_20(8173637) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_20 uid=23340 usec=260 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_21(unfulfilled) for JobId=8173617_21(8173638) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_21 uid=23340 usec=116 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_22(unfulfilled) for JobId=8173617_22(8173639) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_22 uid=23340 usec=119 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_23(unfulfilled) for JobId=8173617_23(8173640) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_23 uid=23340 usec=142 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_24(unfulfilled) for JobId=8173617_24(8173641) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_24 uid=23340 usec=133 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_25(unfulfilled) for JobId=8173617_25(8173642) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_25 uid=23340 usec=126 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_26(unfulfilled) for JobId=8173617_26(8173643) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_26 uid=23340 usec=162 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_27(unfulfilled) for JobId=8173617_27(8173644) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_27 uid=23340 usec=312 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_28(unfulfilled) for JobId=8173617_28(8173645) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_28 uid=23340 usec=148 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_29(unfulfilled) for JobId=8173617_29(8173646) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_29 uid=23340 usec=275 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_30(unfulfilled) for JobId=8173617_30(8173647) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_30 uid=23340 usec=42027 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_31(unfulfilled) for JobId=8173617_31(8173648) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_31 uid=23340 usec=161 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_32(unfulfilled) for JobId=8173617_32(8173649) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_32 uid=23340 usec=156 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_33(unfulfilled) for JobId=8173617_33(8173650) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_33 uid=23340 usec=165 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_34(unfulfilled) for JobId=8173617_34(8173651) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_34 uid=23340 usec=553 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_35(unfulfilled) for JobId=8173617_35(8173652) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_35 uid=23340 usec=134 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_36(unfulfilled) for JobId=8173617_36(8173653) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_36 uid=23340 usec=157 Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_37(unfulfilled) for JobId=8173617_37(8173654) Nov 05 11:40:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_37 uid=23340 usec=127 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_38(unfulfilled) for JobId=8173617_38(8173655) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_38 uid=23340 usec=23091 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_39(unfulfilled) for JobId=8173617_39(8173656) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_39 uid=23340 usec=186 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_40(unfulfilled) for JobId=8173617_40(8173657) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_40 uid=23340 usec=217 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_41(unfulfilled) for JobId=8173617_41(8173658) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_41 uid=23340 usec=135 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_42(unfulfilled) for JobId=8173617_42(8173659) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_42 uid=23340 usec=167 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_43(unfulfilled) for JobId=8173617_43(8173660) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_43 uid=23340 usec=162 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_44(unfulfilled) for JobId=8173617_44(8173661) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_44 uid=23340 usec=122 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_45(unfulfilled) for JobId=8173617_45(8173662) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_45 uid=23340 usec=136 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_46(unfulfilled) for JobId=8173617_46(8173663) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_46 uid=23340 usec=123 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_47(unfulfilled) for JobId=8173617_47(8173664) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_47 uid=23340 usec=163 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_48(unfulfilled) for JobId=8173617_48(8173665) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_48 uid=23340 usec=213 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_49(unfulfilled) for JobId=8173617_49(8173666) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_49 uid=23340 usec=268 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_50(unfulfilled) for JobId=8173617_50(8173667) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_50 uid=23340 usec=285 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_51(unfulfilled) for JobId=8173617_51(8173668) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_51 uid=23340 usec=306 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_52(unfulfilled) for JobId=8173617_52(8173669) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_52 uid=23340 usec=171 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_53(unfulfilled) for JobId=8173617_53(8173670) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_53 uid=23340 usec=157 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_54(unfulfilled) for JobId=8173617_54(8173671) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_54 uid=23340 usec=141 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_55(unfulfilled) for JobId=8173617_55(8173672) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_55 uid=23340 usec=128 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_56(unfulfilled) for JobId=8173617_56(8173673) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_56 uid=23340 usec=125 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_57(unfulfilled) for JobId=8173617_57(8173674) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_57 uid=23340 usec=128 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_58(unfulfilled) for JobId=8173617_58(8173675) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_58 uid=23340 usec=288 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_59(unfulfilled) for JobId=8173617_59(8173676) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_59 uid=23340 usec=135 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_60(unfulfilled) for JobId=8173617_60(8173677) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_60 uid=23340 usec=126 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_61(unfulfilled) for JobId=8173617_61(8173678) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_61 uid=23340 usec=128 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_62(unfulfilled) for JobId=8173617_62(8173679) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_62 uid=23340 usec=195 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_63(unfulfilled) for JobId=8173617_63(8173680) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_63 uid=23340 usec=114 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_64(unfulfilled) for JobId=8173617_64(8173681) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_64 uid=23340 usec=129 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_65(unfulfilled) for JobId=8173617_65(8173682) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_65 uid=23340 usec=123 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_66(unfulfilled) for JobId=8173617_66(8173683) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_66 uid=23340 usec=127 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_67(unfulfilled) for JobId=8173617_67(8173684) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_67 uid=23340 usec=167 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_68(unfulfilled) for JobId=8173617_68(8173685) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_68 uid=23340 usec=125 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_69(unfulfilled) for JobId=8173617_69(8173686) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_69 uid=23340 usec=132 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_70(unfulfilled) for JobId=8173617_70(8173687) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_70 uid=23340 usec=130 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_71(unfulfilled) for JobId=8173617_71(8173688) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_71 uid=23340 usec=121 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_72(unfulfilled) for JobId=8173617_72(8173689) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_72 uid=23340 usec=129 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_73(unfulfilled) for JobId=8173617_73(8173690) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_73 uid=23340 usec=122 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_74(unfulfilled) for JobId=8173617_74(8173691) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_74 uid=23340 usec=126 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_75(unfulfilled) for JobId=8173617_75(8173692) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_75 uid=23340 usec=148 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_76(unfulfilled) for JobId=8173617_76(8173693) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_76 uid=23340 usec=186 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_77(unfulfilled) for JobId=8173617_77(8173694) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_77 uid=23340 usec=309 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_78(unfulfilled) for JobId=8173617_78(8173695) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_78 uid=23340 usec=130 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_79(unfulfilled) for JobId=8173617_79(8173696) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_79 uid=23340 usec=115 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_80(unfulfilled) for JobId=8173617_80(8173697) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_80 uid=23340 usec=156 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_81(unfulfilled) for JobId=8173617_81(8173698) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_81 uid=23340 usec=40534 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_82(unfulfilled) for JobId=8173617_82(8173699) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_82 uid=23340 usec=128 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_83(unfulfilled) for JobId=8173617_83(8173700) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_83 uid=23340 usec=151 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_84(unfulfilled) for JobId=8173617_84(8173701) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_84 uid=23340 usec=126 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_85(unfulfilled) for JobId=8173617_85(8173702) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_85 uid=23340 usec=112 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_86(unfulfilled) for JobId=8173617_86(8173703) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_86 uid=23340 usec=162 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_87(unfulfilled) for JobId=8173617_87(8173704) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_87 uid=23340 usec=143 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_88(unfulfilled) for JobId=8173617_88(8173705) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_88 uid=23340 usec=120 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_89(unfulfilled) for JobId=8173617_89(8173706) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_89 uid=23340 usec=118 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_90(unfulfilled) for JobId=8173617_90(8173707) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_90 uid=23340 usec=112 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_91(unfulfilled) for JobId=8173617_91(8173708) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_91 uid=23340 usec=138 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_92(unfulfilled) for JobId=8173617_92(8173709) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_92 uid=23340 usec=117 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_93(unfulfilled) for JobId=8173617_93(8173710) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_93 uid=23340 usec=142 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_94(unfulfilled) for JobId=8173617_94(8173711) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_94 uid=23340 usec=118 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_95(unfulfilled) for JobId=8173617_95(8173712) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_95 uid=23340 usec=139 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173521_96(unfulfilled) for JobId=8173617_96(8173617) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173617_96 uid=23340 usec=143 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173713 InitPrio=13477 usec=3007 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_1(unfulfilled) for JobId=8173713_1(8173714) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_1 uid=23340 usec=191 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_2(unfulfilled) for JobId=8173713_2(8173715) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_2 uid=23340 usec=156 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_3(unfulfilled) for JobId=8173713_3(8173716) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_3 uid=23340 usec=133 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_4(unfulfilled) for JobId=8173713_4(8173717) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_4 uid=23340 usec=138 Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_5(unfulfilled) for JobId=8173713_5(8173718) Nov 05 11:41:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_5 uid=23340 usec=165 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_6(unfulfilled) for JobId=8173713_6(8173719) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_6 uid=23340 usec=45325 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_7(unfulfilled) for JobId=8173713_7(8173720) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_7 uid=23340 usec=230 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_8(unfulfilled) for JobId=8173713_8(8173721) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_8 uid=23340 usec=173 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_9(unfulfilled) for JobId=8173713_9(8173722) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_9 uid=23340 usec=308 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_10(unfulfilled) for JobId=8173713_10(8173723) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_10 uid=23340 usec=140 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_11(unfulfilled) for JobId=8173713_11(8173724) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_11 uid=23340 usec=163 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_12(unfulfilled) for JobId=8173713_12(8173725) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_12 uid=23340 usec=136 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_13(unfulfilled) for JobId=8173713_13(8173726) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_13 uid=23340 usec=143 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_14(unfulfilled) for JobId=8173713_14(8173727) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_14 uid=23340 usec=138 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_15(unfulfilled) for JobId=8173713_15(8173728) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_15 uid=23340 usec=179 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_16(unfulfilled) for JobId=8173713_16(8173729) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_16 uid=23340 usec=304 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_17(unfulfilled) for JobId=8173713_17(8173730) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_17 uid=23340 usec=175 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_18(unfulfilled) for JobId=8173713_18(8173731) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_18 uid=23340 usec=159 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_19(unfulfilled) for JobId=8173713_19(8173732) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_19 uid=23340 usec=132 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_20(unfulfilled) for JobId=8173713_20(8173733) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_20 uid=23340 usec=125 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_21(unfulfilled) for JobId=8173713_21(8173734) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_21 uid=23340 usec=134 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_22(unfulfilled) for JobId=8173713_22(8173735) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_22 uid=23340 usec=149 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_23(unfulfilled) for JobId=8173713_23(8173736) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_23 uid=23340 usec=127 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_24(unfulfilled) for JobId=8173713_24(8173737) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_24 uid=23340 usec=135 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_25(unfulfilled) for JobId=8173713_25(8173738) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_25 uid=23340 usec=132 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_26(unfulfilled) for JobId=8173713_26(8173739) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_26 uid=23340 usec=151 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_27(unfulfilled) for JobId=8173713_27(8173740) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_27 uid=23340 usec=324 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_28(unfulfilled) for JobId=8173713_28(8173741) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_28 uid=23340 usec=139 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_29(unfulfilled) for JobId=8173713_29(8173742) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_29 uid=23340 usec=122 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_30(unfulfilled) for JobId=8173713_30(8173743) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_30 uid=23340 usec=46582 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_31(unfulfilled) for JobId=8173713_31(8173744) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_31 uid=23340 usec=157 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_32(unfulfilled) for JobId=8173713_32(8173745) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_32 uid=23340 usec=181 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_33(unfulfilled) for JobId=8173713_33(8173746) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_33 uid=23340 usec=133 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_34(unfulfilled) for JobId=8173713_34(8173747) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_34 uid=23340 usec=118 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_35(unfulfilled) for JobId=8173713_35(8173748) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_35 uid=23340 usec=112 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_36(unfulfilled) for JobId=8173713_36(8173749) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_36 uid=23340 usec=135 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_37(unfulfilled) for JobId=8173713_37(8173750) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_37 uid=23340 usec=125 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_38(unfulfilled) for JobId=8173713_38(8173751) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_38 uid=23340 usec=120 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_39(unfulfilled) for JobId=8173713_39(8173752) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_39 uid=23340 usec=115 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_40(unfulfilled) for JobId=8173713_40(8173753) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_40 uid=23340 usec=117 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_41(unfulfilled) for JobId=8173713_41(8173754) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_41 uid=23340 usec=111 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_42(unfulfilled) for JobId=8173713_42(8173755) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_42 uid=23340 usec=136 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_43(unfulfilled) for JobId=8173713_43(8173756) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_43 uid=23340 usec=120 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_44(unfulfilled) for JobId=8173713_44(8173757) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_44 uid=23340 usec=110 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_45(unfulfilled) for JobId=8173713_45(8173758) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_45 uid=23340 usec=111 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_46(unfulfilled) for JobId=8173713_46(8173759) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_46 uid=23340 usec=132 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_47(unfulfilled) for JobId=8173713_47(8173760) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_47 uid=23340 usec=120 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_48(unfulfilled) for JobId=8173713_48(8173761) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_48 uid=23340 usec=112 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_49(unfulfilled) for JobId=8173713_49(8173762) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_49 uid=23340 usec=211 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_50(unfulfilled) for JobId=8173713_50(8173763) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_50 uid=23340 usec=108 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_51(unfulfilled) for JobId=8173713_51(8173764) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_51 uid=23340 usec=110 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_52(unfulfilled) for JobId=8173713_52(8173765) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_52 uid=23340 usec=144 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_53(unfulfilled) for JobId=8173713_53(8173766) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_53 uid=23340 usec=146 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_54(unfulfilled) for JobId=8173713_54(8173767) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_54 uid=23340 usec=225 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_55(unfulfilled) for JobId=8173713_55(8173768) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_55 uid=23340 usec=111 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_56(unfulfilled) for JobId=8173713_56(8173769) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_56 uid=23340 usec=250 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_57(unfulfilled) for JobId=8173713_57(8173770) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_57 uid=23340 usec=109 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_58(unfulfilled) for JobId=8173713_58(8173771) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_58 uid=23340 usec=162 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_59(unfulfilled) for JobId=8173713_59(8173772) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_59 uid=23340 usec=159 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_60(unfulfilled) for JobId=8173713_60(8173773) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_60 uid=23340 usec=129 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_61(unfulfilled) for JobId=8173713_61(8173774) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_61 uid=23340 usec=118 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_62(unfulfilled) for JobId=8173713_62(8173775) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_62 uid=23340 usec=141 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_63(unfulfilled) for JobId=8173713_63(8173776) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_63 uid=23340 usec=130 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_64(unfulfilled) for JobId=8173713_64(8173777) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_64 uid=23340 usec=125 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_65(unfulfilled) for JobId=8173713_65(8173778) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_65 uid=23340 usec=139 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_66(unfulfilled) for JobId=8173713_66(8173779) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_66 uid=23340 usec=120 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_67(unfulfilled) for JobId=8173713_67(8173780) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_67 uid=23340 usec=128 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_68(unfulfilled) for JobId=8173713_68(8173781) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_68 uid=23340 usec=120 Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_69(unfulfilled) for JobId=8173713_69(8173782) Nov 05 11:41:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_69 uid=23340 usec=117 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_70(unfulfilled) for JobId=8173713_70(8173783) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_70 uid=23340 usec=119 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_71(unfulfilled) for JobId=8173713_71(8173784) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_71 uid=23340 usec=21725 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_72(unfulfilled) for JobId=8173713_72(8173785) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_72 uid=23340 usec=174 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_73(unfulfilled) for JobId=8173713_73(8173786) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_73 uid=23340 usec=322 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_74(unfulfilled) for JobId=8173713_74(8173787) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_74 uid=23340 usec=132 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_75(unfulfilled) for JobId=8173713_75(8173788) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_75 uid=23340 usec=116 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_76(unfulfilled) for JobId=8173713_76(8173789) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_76 uid=23340 usec=140 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_77(unfulfilled) for JobId=8173713_77(8173790) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_77 uid=23340 usec=163 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_78(unfulfilled) for JobId=8173713_78(8173791) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_78 uid=23340 usec=305 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_79(unfulfilled) for JobId=8173713_79(8173792) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_79 uid=23340 usec=38398 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_80(unfulfilled) for JobId=8173713_80(8173793) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_80 uid=23340 usec=138 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_81(unfulfilled) for JobId=8173713_81(8173794) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_81 uid=23340 usec=124 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_82(unfulfilled) for JobId=8173713_82(8173795) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_82 uid=23340 usec=147 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_83(unfulfilled) for JobId=8173713_83(8173796) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_83 uid=23340 usec=127 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_84(unfulfilled) for JobId=8173713_84(8173797) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_84 uid=23340 usec=124 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_85(unfulfilled) for JobId=8173713_85(8173798) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_85 uid=23340 usec=119 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_86(unfulfilled) for JobId=8173713_86(8173799) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_86 uid=23340 usec=138 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_87(unfulfilled) for JobId=8173713_87(8173800) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_87 uid=23340 usec=129 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_88(unfulfilled) for JobId=8173713_88(8173801) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_88 uid=23340 usec=120 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_89(unfulfilled) for JobId=8173713_89(8173802) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_89 uid=23340 usec=116 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_90(unfulfilled) for JobId=8173713_90(8173803) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_90 uid=23340 usec=117 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_91(unfulfilled) for JobId=8173713_91(8173804) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_91 uid=23340 usec=123 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_92(unfulfilled) for JobId=8173713_92(8173805) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_92 uid=23340 usec=135 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_93(unfulfilled) for JobId=8173713_93(8173806) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_93 uid=23340 usec=125 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_94(unfulfilled) for JobId=8173713_94(8173807) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_94 uid=23340 usec=134 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_95(unfulfilled) for JobId=8173713_95(8173808) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_95 uid=23340 usec=126 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173617_96(unfulfilled) for JobId=8173713_96(8173713) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173713_96 uid=23340 usec=118 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173809 InitPrio=13477 usec=2430 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_1(unfulfilled) for JobId=8173809_1(8173810) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_1 uid=23340 usec=173 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_2(unfulfilled) for JobId=8173809_2(8173811) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_2 uid=23340 usec=119 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_3(unfulfilled) for JobId=8173809_3(8173812) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_3 uid=23340 usec=133 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_4(unfulfilled) for JobId=8173809_4(8173813) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_4 uid=23340 usec=127 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_5(unfulfilled) for JobId=8173809_5(8173814) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_5 uid=23340 usec=112 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_6(unfulfilled) for JobId=8173809_6(8173815) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_6 uid=23340 usec=114 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_7(unfulfilled) for JobId=8173809_7(8173816) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_7 uid=23340 usec=114 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_8(unfulfilled) for JobId=8173809_8(8173817) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_8 uid=23340 usec=138 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_9(unfulfilled) for JobId=8173809_9(8173818) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_9 uid=23340 usec=118 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_10(unfulfilled) for JobId=8173809_10(8173819) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_10 uid=23340 usec=121 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_11(unfulfilled) for JobId=8173809_11(8173820) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_11 uid=23340 usec=184 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_12(unfulfilled) for JobId=8173809_12(8173821) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_12 uid=23340 usec=176 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_13(unfulfilled) for JobId=8173809_13(8173822) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_13 uid=23340 usec=115 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_14(unfulfilled) for JobId=8173809_14(8173823) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_14 uid=23340 usec=116 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_15(unfulfilled) for JobId=8173809_15(8173824) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_15 uid=23340 usec=126 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_16(unfulfilled) for JobId=8173809_16(8173825) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_16 uid=23340 usec=131 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_17(unfulfilled) for JobId=8173809_17(8173826) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_17 uid=23340 usec=116 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_18(unfulfilled) for JobId=8173809_18(8173827) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_18 uid=23340 usec=148 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_19(unfulfilled) for JobId=8173809_19(8173828) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_19 uid=23340 usec=121 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_20(unfulfilled) for JobId=8173809_20(8173829) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_20 uid=23340 usec=301 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_21(unfulfilled) for JobId=8173809_21(8173830) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_21 uid=23340 usec=119 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_22(unfulfilled) for JobId=8173809_22(8173831) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_22 uid=23340 usec=124 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_23(unfulfilled) for JobId=8173809_23(8173832) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_23 uid=23340 usec=115 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_24(unfulfilled) for JobId=8173809_24(8173833) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_24 uid=23340 usec=120 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_25(unfulfilled) for JobId=8173809_25(8173834) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_25 uid=23340 usec=116 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_26(unfulfilled) for JobId=8173809_26(8173835) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_26 uid=23340 usec=273 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_27(unfulfilled) for JobId=8173809_27(8173836) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_27 uid=23340 usec=120 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_28(unfulfilled) for JobId=8173809_28(8173837) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_28 uid=23340 usec=325 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_29(unfulfilled) for JobId=8173809_29(8173838) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_29 uid=23340 usec=123 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_30(unfulfilled) for JobId=8173809_30(8173839) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_30 uid=23340 usec=319 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_31(unfulfilled) for JobId=8173809_31(8173840) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_31 uid=23340 usec=171 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_32(unfulfilled) for JobId=8173809_32(8173841) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_32 uid=23340 usec=113 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_33(unfulfilled) for JobId=8173809_33(8173842) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_33 uid=23340 usec=46227 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_34(unfulfilled) for JobId=8173809_34(8173843) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_34 uid=23340 usec=158 Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_35(unfulfilled) for JobId=8173809_35(8173844) Nov 05 11:41:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_35 uid=23340 usec=121 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_36(unfulfilled) for JobId=8173809_36(8173845) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_36 uid=23340 usec=18726 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_37(unfulfilled) for JobId=8173809_37(8173846) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_37 uid=23340 usec=143 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_38(unfulfilled) for JobId=8173809_38(8173847) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_38 uid=23340 usec=232 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_39(unfulfilled) for JobId=8173809_39(8173848) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_39 uid=23340 usec=182 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_40(unfulfilled) for JobId=8173809_40(8173849) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_40 uid=23340 usec=131 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_41(unfulfilled) for JobId=8173809_41(8173850) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_41 uid=23340 usec=124 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_42(unfulfilled) for JobId=8173809_42(8173851) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_42 uid=23340 usec=115 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_43(unfulfilled) for JobId=8173809_43(8173852) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_43 uid=23340 usec=115 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_44(unfulfilled) for JobId=8173809_44(8173853) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_44 uid=23340 usec=172 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_45(unfulfilled) for JobId=8173809_45(8173854) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_45 uid=23340 usec=143 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_46(unfulfilled) for JobId=8173809_46(8173855) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_46 uid=23340 usec=183 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_47(unfulfilled) for JobId=8173809_47(8173856) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_47 uid=23340 usec=129 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_48(unfulfilled) for JobId=8173809_48(8173857) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_48 uid=23340 usec=156 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_49(unfulfilled) for JobId=8173809_49(8173858) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_49 uid=23340 usec=150 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_50(unfulfilled) for JobId=8173809_50(8173859) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_50 uid=23340 usec=113 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_51(unfulfilled) for JobId=8173809_51(8173860) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_51 uid=23340 usec=111 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_52(unfulfilled) for JobId=8173809_52(8173861) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_52 uid=23340 usec=164 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_53(unfulfilled) for JobId=8173809_53(8173862) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_53 uid=23340 usec=120 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_54(unfulfilled) for JobId=8173809_54(8173863) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_54 uid=23340 usec=172 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_55(unfulfilled) for JobId=8173809_55(8173864) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_55 uid=23340 usec=146 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_56(unfulfilled) for JobId=8173809_56(8173865) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_56 uid=23340 usec=118 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_57(unfulfilled) for JobId=8173809_57(8173866) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_57 uid=23340 usec=112 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_58(unfulfilled) for JobId=8173809_58(8173867) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_58 uid=23340 usec=112 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_59(unfulfilled) for JobId=8173809_59(8173868) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_59 uid=23340 usec=110 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_60(unfulfilled) for JobId=8173809_60(8173869) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_60 uid=23340 usec=122 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_61(unfulfilled) for JobId=8173809_61(8173870) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_61 uid=23340 usec=204 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_62(unfulfilled) for JobId=8173809_62(8173871) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_62 uid=23340 usec=173 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_63(unfulfilled) for JobId=8173809_63(8173872) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_63 uid=23340 usec=169 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_64(unfulfilled) for JobId=8173809_64(8173873) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_64 uid=23340 usec=123 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_65(unfulfilled) for JobId=8173809_65(8173874) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_65 uid=23340 usec=114 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_66(unfulfilled) for JobId=8173809_66(8173875) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_66 uid=23340 usec=111 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_67(unfulfilled) for JobId=8173809_67(8173876) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_67 uid=23340 usec=121 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_68(unfulfilled) for JobId=8173809_68(8173877) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_68 uid=23340 usec=110 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_69(unfulfilled) for JobId=8173809_69(8173878) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_69 uid=23340 usec=113 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_70(unfulfilled) for JobId=8173809_70(8173879) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_70 uid=23340 usec=111 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_71(unfulfilled) for JobId=8173809_71(8173880) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_71 uid=23340 usec=111 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_72(unfulfilled) for JobId=8173809_72(8173881) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_72 uid=23340 usec=113 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_73(unfulfilled) for JobId=8173809_73(8173882) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_73 uid=23340 usec=115 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_74(unfulfilled) for JobId=8173809_74(8173883) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_74 uid=23340 usec=121 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_75(unfulfilled) for JobId=8173809_75(8173884) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_75 uid=23340 usec=113 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_76(unfulfilled) for JobId=8173809_76(8173885) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_76 uid=23340 usec=112 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_77(unfulfilled) for JobId=8173809_77(8173886) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_77 uid=23340 usec=110 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_78(unfulfilled) for JobId=8173809_78(8173887) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_78 uid=23340 usec=111 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_79(unfulfilled) for JobId=8173809_79(8173888) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_79 uid=23340 usec=110 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_80(unfulfilled) for JobId=8173809_80(8173889) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_80 uid=23340 usec=110 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_81(unfulfilled) for JobId=8173809_81(8173890) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_81 uid=23340 usec=119 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_82(unfulfilled) for JobId=8173809_82(8173891) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_82 uid=23340 usec=112 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_83(unfulfilled) for JobId=8173809_83(8173892) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_83 uid=23340 usec=41072 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8151(8172956) WEXITSTATUS 1 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8151(8172956) done Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_84(unfulfilled) for JobId=8173809_84(8173893) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_84 uid=23340 usec=141 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_85(unfulfilled) for JobId=8173809_85(8173894) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_85 uid=23340 usec=126 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_86(unfulfilled) for JobId=8173809_86(8173895) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_86 uid=23340 usec=121 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_87(unfulfilled) for JobId=8173809_87(8173896) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_87 uid=23340 usec=136 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_88(unfulfilled) for JobId=8173809_88(8173897) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_88 uid=23340 usec=138 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_89(unfulfilled) for JobId=8173809_89(8173898) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_89 uid=23340 usec=130 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_90(unfulfilled) for JobId=8173809_90(8173899) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_90 uid=23340 usec=128 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_91(unfulfilled) for JobId=8173809_91(8173900) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_91 uid=23340 usec=184 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_92(unfulfilled) for JobId=8173809_92(8173901) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_92 uid=23340 usec=140 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_93(unfulfilled) for JobId=8173809_93(8173902) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_93 uid=23340 usec=136 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_94(unfulfilled) for JobId=8173809_94(8173903) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_94 uid=23340 usec=124 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_95(unfulfilled) for JobId=8173809_95(8173904) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_95 uid=23340 usec=140 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173713_96(unfulfilled) for JobId=8173809_96(8173809) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173809_96 uid=23340 usec=115 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8173905 InitPrio=13477 usec=2496 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_1(unfulfilled) for JobId=8173905_1(8173906) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_1 uid=23340 usec=130 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_2(unfulfilled) for JobId=8173905_2(8173907) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_2 uid=23340 usec=201 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_3(unfulfilled) for JobId=8173905_3(8173908) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_3 uid=23340 usec=176 Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_4(unfulfilled) for JobId=8173905_4(8173909) Nov 05 11:41:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_4 uid=23340 usec=129 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_5(unfulfilled) for JobId=8173905_5(8173910) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_5 uid=23340 usec=142669 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_6(unfulfilled) for JobId=8173905_6(8173911) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_6 uid=23340 usec=182 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_7(unfulfilled) for JobId=8173905_7(8173912) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_7 uid=23340 usec=128 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_8(unfulfilled) for JobId=8173905_8(8173913) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_8 uid=23340 usec=120 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_9(unfulfilled) for JobId=8173905_9(8173914) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_9 uid=23340 usec=117 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_10(unfulfilled) for JobId=8173905_10(8173915) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_10 uid=23340 usec=333 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_11(unfulfilled) for JobId=8173905_11(8173916) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_11 uid=23340 usec=114 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_12(unfulfilled) for JobId=8173905_12(8173917) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_12 uid=23340 usec=110 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_13(unfulfilled) for JobId=8173905_13(8173918) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_13 uid=23340 usec=113 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_14(unfulfilled) for JobId=8173905_14(8173919) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_14 uid=23340 usec=114 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_15(unfulfilled) for JobId=8173905_15(8173920) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_15 uid=23340 usec=297 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_16(unfulfilled) for JobId=8173905_16(8173921) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_16 uid=23340 usec=112 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_17(unfulfilled) for JobId=8173905_17(8173922) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_17 uid=23340 usec=113 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_18(unfulfilled) for JobId=8173905_18(8173923) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_18 uid=23340 usec=108 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_19(unfulfilled) for JobId=8173905_19(8173924) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_19 uid=23340 usec=112 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_20(unfulfilled) for JobId=8173905_20(8173925) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_20 uid=23340 usec=118 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_21(unfulfilled) for JobId=8173905_21(8173926) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_21 uid=23340 usec=133 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_22(unfulfilled) for JobId=8173905_22(8173927) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_22 uid=23340 usec=127 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_23(unfulfilled) for JobId=8173905_23(8173928) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_23 uid=23340 usec=134 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_24(unfulfilled) for JobId=8173905_24(8173929) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_24 uid=23340 usec=113 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_25(unfulfilled) for JobId=8173905_25(8173930) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_25 uid=23340 usec=41108 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_26(unfulfilled) for JobId=8173905_26(8173931) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_26 uid=23340 usec=140 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_27(unfulfilled) for JobId=8173905_27(8173932) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_27 uid=23340 usec=116 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_28(unfulfilled) for JobId=8173905_28(8173933) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_28 uid=23340 usec=115 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_29(unfulfilled) for JobId=8173905_29(8173934) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_29 uid=23340 usec=119 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_30(unfulfilled) for JobId=8173905_30(8173935) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_30 uid=23340 usec=116 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_31(unfulfilled) for JobId=8173905_31(8173936) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_31 uid=23340 usec=120 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_32(unfulfilled) for JobId=8173905_32(8173937) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_32 uid=23340 usec=115 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_33(unfulfilled) for JobId=8173905_33(8173938) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_33 uid=23340 usec=121 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_34(unfulfilled) for JobId=8173905_34(8173939) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_34 uid=23340 usec=112 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_35(unfulfilled) for JobId=8173905_35(8173940) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_35 uid=23340 usec=120 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_36(unfulfilled) for JobId=8173905_36(8173941) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_36 uid=23340 usec=112 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_37(unfulfilled) for JobId=8173905_37(8173942) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_37 uid=23340 usec=111 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_38(unfulfilled) for JobId=8173905_38(8173943) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_38 uid=23340 usec=124 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_39(unfulfilled) for JobId=8173905_39(8173944) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_39 uid=23340 usec=114 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_40(unfulfilled) for JobId=8173905_40(8173945) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_40 uid=23340 usec=114 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_41(unfulfilled) for JobId=8173905_41(8173946) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_41 uid=23340 usec=113 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_42(unfulfilled) for JobId=8173905_42(8173947) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_42 uid=23340 usec=113 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_43(unfulfilled) for JobId=8173905_43(8173948) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_43 uid=23340 usec=115 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_44(unfulfilled) for JobId=8173905_44(8173949) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_44 uid=23340 usec=278 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_45(unfulfilled) for JobId=8173905_45(8173950) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_45 uid=23340 usec=168 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_46(unfulfilled) for JobId=8173905_46(8173951) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_46 uid=23340 usec=114 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_47(unfulfilled) for JobId=8173905_47(8173952) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_47 uid=23340 usec=123 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_48(unfulfilled) for JobId=8173905_48(8173953) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_48 uid=23340 usec=112 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_49(unfulfilled) for JobId=8173905_49(8173954) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_49 uid=23340 usec=197 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_50(unfulfilled) for JobId=8173905_50(8173955) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_50 uid=23340 usec=114 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_51(unfulfilled) for JobId=8173905_51(8173956) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_51 uid=23340 usec=112 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_52(unfulfilled) for JobId=8173905_52(8173957) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_52 uid=23340 usec=118 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_53(unfulfilled) for JobId=8173905_53(8173958) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_53 uid=23340 usec=113 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_54(unfulfilled) for JobId=8173905_54(8173959) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_54 uid=23340 usec=111 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_55(unfulfilled) for JobId=8173905_55(8173960) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_55 uid=23340 usec=109 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_56(unfulfilled) for JobId=8173905_56(8173961) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_56 uid=23340 usec=113 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_57(unfulfilled) for JobId=8173905_57(8173962) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_57 uid=23340 usec=109 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_58(unfulfilled) for JobId=8173905_58(8173963) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_58 uid=23340 usec=110 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_59(unfulfilled) for JobId=8173905_59(8173964) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_59 uid=23340 usec=110 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_60(unfulfilled) for JobId=8173905_60(8173965) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_60 uid=23340 usec=122 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_61(unfulfilled) for JobId=8173905_61(8173966) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_61 uid=23340 usec=119 Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_62(unfulfilled) for JobId=8173905_62(8173967) Nov 05 11:41:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_62 uid=23340 usec=117 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_63(unfulfilled) for JobId=8173905_63(8173968) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_63 uid=23340 usec=20682 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_64(unfulfilled) for JobId=8173905_64(8173969) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_64 uid=23340 usec=166 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_65(unfulfilled) for JobId=8173905_65(8173970) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_65 uid=23340 usec=121 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_66(unfulfilled) for JobId=8173905_66(8173971) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_66 uid=23340 usec=115 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_67(unfulfilled) for JobId=8173905_67(8173972) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_67 uid=23340 usec=115 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_68(unfulfilled) for JobId=8173905_68(8173973) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_68 uid=23340 usec=111 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_69(unfulfilled) for JobId=8173905_69(8173974) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_69 uid=23340 usec=111 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_70(unfulfilled) for JobId=8173905_70(8173975) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_70 uid=23340 usec=110 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_71(unfulfilled) for JobId=8173905_71(8173976) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_71 uid=23340 usec=109 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_72(unfulfilled) for JobId=8173905_72(8173977) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_72 uid=23340 usec=196 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_73(unfulfilled) for JobId=8173905_73(8173978) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_73 uid=23340 usec=48563 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_74(unfulfilled) for JobId=8173905_74(8173979) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_74 uid=23340 usec=140 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_75(unfulfilled) for JobId=8173905_75(8173980) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_75 uid=23340 usec=134 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_76(unfulfilled) for JobId=8173905_76(8173981) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_76 uid=23340 usec=132 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_77(unfulfilled) for JobId=8173905_77(8173982) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_77 uid=23340 usec=123 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_78(unfulfilled) for JobId=8173905_78(8173983) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_78 uid=23340 usec=118 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_79(unfulfilled) for JobId=8173905_79(8173984) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_79 uid=23340 usec=116 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_80(unfulfilled) for JobId=8173905_80(8173985) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_80 uid=23340 usec=116 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_81(unfulfilled) for JobId=8173905_81(8173986) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_81 uid=23340 usec=131 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_82(unfulfilled) for JobId=8173905_82(8173987) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_82 uid=23340 usec=141 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_83(unfulfilled) for JobId=8173905_83(8173988) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_83 uid=23340 usec=141 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_84(unfulfilled) for JobId=8173905_84(8173989) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_84 uid=23340 usec=122 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_85(unfulfilled) for JobId=8173905_85(8173990) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_85 uid=23340 usec=119 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_86(unfulfilled) for JobId=8173905_86(8173991) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_86 uid=23340 usec=120 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_87(unfulfilled) for JobId=8173905_87(8173992) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_87 uid=23340 usec=119 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_88(unfulfilled) for JobId=8173905_88(8173993) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_88 uid=23340 usec=115 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_89(unfulfilled) for JobId=8173905_89(8173994) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_89 uid=23340 usec=115 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_90(unfulfilled) for JobId=8173905_90(8173995) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_90 uid=23340 usec=126 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_91(unfulfilled) for JobId=8173905_91(8173996) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_91 uid=23340 usec=118 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_92(unfulfilled) for JobId=8173905_92(8173997) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_92 uid=23340 usec=121 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_93(unfulfilled) for JobId=8173905_93(8173998) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_93 uid=23340 usec=121 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_94(unfulfilled) for JobId=8173905_94(8173999) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_94 uid=23340 usec=127 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_95(unfulfilled) for JobId=8173905_95(8174000) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_95 uid=23340 usec=129 Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8173809_96(unfulfilled) for JobId=8173905_96(8173905) Nov 05 11:41:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8173905_96 uid=23340 usec=108 Nov 05 11:41:19 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8168530 Nov 05 11:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8110(8172785) WEXITSTATUS 1 Nov 05 11:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8110(8172785) done Nov 05 11:41:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_50(7572217) Nov 05 11:41:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_653(7572822) Nov 05 11:41:50 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_931(7573100) Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2578(8152845) WEXITSTATUS 0 Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2578(8152845) done Nov 05 11:41:57 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:41:58 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:41:58 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 11:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160525 WEXITSTATUS 0 Nov 05 11:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8160525 done Nov 05 11:42:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7833(8169550) WEXITSTATUS 1 Nov 05 11:42:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7833(8169550) done Nov 05 11:42:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2267(8165098) WEXITSTATUS 0 Nov 05 11:42:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2267(8165098) done Nov 05 11:42:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7892(8170010) WEXITSTATUS 1 Nov 05 11:42:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7892(8170010) done Nov 05 11:42:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 11:42:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7874(8169906) WEXITSTATUS 1 Nov 05 11:42:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7874(8169906) done Nov 05 11:42:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7812(8169414) WEXITSTATUS 1 Nov 05 11:42:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7812(8169414) done Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8161(8174001) in scavenge on r801u06n01 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8162(8174002) in scavenge on r802u32n03 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8163(8174003) in scavenge on r802u32n04 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8164(8174004) in scavenge on r911u13n04 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8165(8174005) in scavenge on r911u24n02 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8166(8174006) in scavenge on r815u03n10 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8167(8174007) in scavenge on r816u03n12 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8168(8174008) in scavenge on r816u15n12 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8169(8174009) in scavenge on r807u08n01 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8170(8174010) in scavenge on r909u13n01 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_344(7571309) in scavenge on r802u11n02 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_793(7572962) in scavenge on r802u11n03 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1146(7573315) in scavenge on r911u13n02 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1082(8115351) in scavenge on r911u13n03 Nov 05 11:42:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_724(7572893) in scavenge on r906u35n01 Nov 05 11:42:20 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_1583(8150562) Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_1(8174011) in scavenge on r908u28n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_2(8174012) in scavenge on r907u01n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_3(8174013) in scavenge on r907u01n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_4(8174014) in scavenge on r907u01n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_5(8174015) in scavenge on r907u01n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_6(8174016) in scavenge on r907u01n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_7(8174017) in scavenge on r907u01n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_8(8174018) in scavenge on r907u05n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_9(8174019) in scavenge on r907u05n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_10(8174020) in scavenge on r907u05n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_11(8174021) in scavenge on r907u05n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_12(8174022) in scavenge on r907u05n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_13(8174023) in scavenge on r907u05n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_14(8174024) in scavenge on r907u05n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_15(8174025) in scavenge on r907u05n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_16(8174026) in scavenge on r802u30n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_17(8174027) in scavenge on r802u30n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_18(8174028) in scavenge on r915u35n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_19(8174029) in scavenge on r917u10n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_20(8174030) in scavenge on r801u04n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_21(8174031) in scavenge on r801u04n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_22(8174032) in scavenge on r801u11n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_23(8174033) in scavenge on r911u09n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_24(8174034) in scavenge on r911u09n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_25(8174035) in scavenge on r911u28n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_26(8174036) in scavenge on r911u28n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_27(8174037) in scavenge on r911u28n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_28(8174038) in scavenge on r911u28n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_29(8174039) in scavenge on r911u28n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_30(8174040) in scavenge on r815u03n10 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_31(8174041) in scavenge on r815u03n10 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_32(8174042) in scavenge on r816u03n09 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_33(8174043) in scavenge on r816u23n08 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_34(8174044) in scavenge on r903u34n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_35(8174045) in scavenge on r808u14n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_36(8174046) in scavenge on r808u17n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_37(8174047) in scavenge on r4513u23n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_38(8174048) in scavenge on r902u07n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_39(8174049) in scavenge on r902u07n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_40(8174050) in scavenge on r907u32n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_41(8174051) in scavenge on r906u28n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_42(8174052) in scavenge on r807u03n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_43(8174053) in scavenge on r912u28n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_44(8174054) in scavenge on r805u11n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_45(8174055) in scavenge on r805u23n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_46(8174056) in scavenge on r909u07n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_47(8174057) in scavenge on r909u10n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_48(8174058) in scavenge on r806u06n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_49(8174059) in scavenge on r806u06n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_50(8174060) in scavenge on r806u09n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_51(8174061) in scavenge on r806u11n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_52(8174062) in scavenge on r806u11n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_53(8174063) in scavenge on r806u11n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_54(8174064) in scavenge on r806u11n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_55(8174065) in scavenge on r918u05n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_56(8174066) in scavenge on r919u01n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_57(8174067) in scavenge on r919u01n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_58(8174068) in scavenge on r919u01n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_59(8174069) in scavenge on r919u01n03 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:118 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_60(8174070) in scavenge on r919u09n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_61(8174071) in scavenge on r815u35n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_62(8174072) in scavenge on r816u09n06 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_63(8174073) in scavenge on r816u09n06 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_64(8174074) in scavenge on r816u09n06 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_65(8174075) in scavenge on r816u15n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_66(8174076) in scavenge on r815u03n08 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_67(8174077) in scavenge on r815u09n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_68(8174078) in scavenge on r815u09n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_69(8174079) in scavenge on r815u15n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_70(8174080) in scavenge on r816u29n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_71(8174081) in scavenge on r816u29n05 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_72(8174082) in scavenge on r816u29n06 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_73(8174083) in scavenge on r816u35n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_74(8174084) in scavenge on r816u35n04 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_75(8174085) in scavenge on r816u35n05 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_76(8174086) in scavenge on r908u24n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_77(8174087) in scavenge on r908u24n02 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_78(8174088) in scavenge on r806u35n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_79(8174089) in scavenge on r806u35n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_80(8174090) in scavenge on r806u35n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_81(8174091) in scavenge on r806u37n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_82(8174092) in scavenge on r915u09n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_83(8174093) in scavenge on r915u13n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_84(8174094) in scavenge on r915u13n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_85(8174095) in scavenge on r805u32n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_86(8174096) in scavenge on r805u32n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_87(8174097) in scavenge on r805u34n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_88(8174098) in scavenge on r805u34n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_89(8174099) in scavenge on r805u34n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_90(8174100) in scavenge on r805u36n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173040_91(8174101) in scavenge on r805u36n01 Nov 05 11:42:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2962(8174102) in scavenge on r815u09n10 Nov 05 11:42:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7706(8168509) WEXITSTATUS 1 Nov 05 11:42:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7706(8168509) done Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_39(8174049) WEXITSTATUS 1 Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_39(8174049) done Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_38(8174048) WEXITSTATUS 1 Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_38(8174048) done Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_37(8174047) WEXITSTATUS 1 Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_37(8174047) done Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_36(8174046) WEXITSTATUS 1 Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_36(8174046) done Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_65(8174075) WEXITSTATUS 1 Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_65(8174075) done Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_34(8174044) WEXITSTATUS 1 Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_34(8174044) done Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_33(8174043) WEXITSTATUS 1 Nov 05 11:42:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_33(8174043) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_69(8174079) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_69(8174079) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_35(8174045) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_35(8174045) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_68(8174078) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_68(8174078) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_67(8174077) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_67(8174077) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_30(8174040) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_30(8174040) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_31(8174041) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_31(8174041) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_70(8174080) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_70(8174080) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_61(8174071) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_61(8174071) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_32(8174042) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_32(8174042) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_63(8174073) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_63(8174073) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_64(8174074) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_64(8174074) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_62(8174072) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_62(8174072) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_75(8174085) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_75(8174085) done Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_11(8174021) WEXITSTATUS 1 Nov 05 11:42:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_11(8174021) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_10(8174020) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_10(8174020) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_72(8174082) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_72(8174082) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_12(8174022) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_12(8174022) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_13(8174023) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_13(8174023) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_82(8174092) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_82(8174092) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_55(8174065) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_55(8174065) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_73(8174083) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_73(8174083) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_6(8174016) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_6(8174016) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_7(8174017) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_7(8174017) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_4(8174014) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_4(8174014) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_5(8174015) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_5(8174015) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_47(8174057) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_47(8174057) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_9(8174019) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_9(8174019) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_71(8174081) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_71(8174081) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_46(8174056) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_46(8174056) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_8(8174018) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_8(8174018) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_83(8174093) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_83(8174093) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_84(8174094) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_84(8174094) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_66(8174076) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_66(8174076) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_2(8174012) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_2(8174012) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_3(8174013) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_3(8174013) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_74(8174084) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_74(8174084) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_14(8174024) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_14(8174024) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_15(8174025) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_15(8174025) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174103 InitPrio=29089 usec=7836 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_57(8174067) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_57(8174067) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_59(8174069) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_59(8174069) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_58(8174068) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_58(8174068) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_18(8174028) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_18(8174028) done Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_24(8174034) WEXITSTATUS 1 Nov 05 11:42:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_24(8174034) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_1(8174011) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_1(8174011) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_77(8174087) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_77(8174087) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_76(8174086) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_76(8174086) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_19(8174029) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_19(8174029) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_45(8174055) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_45(8174055) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_43(8174053) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_43(8174053) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_41(8174051) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_41(8174051) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_23(8174033) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_23(8174033) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_26(8174036) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_26(8174036) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_25(8174035) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_25(8174035) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_22(8174032) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_22(8174032) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_56(8174066) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_56(8174066) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_50(8174060) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_50(8174060) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_51(8174061) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_51(8174061) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_60(8174070) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_60(8174070) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_54(8174064) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_54(8174064) done Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_16(8174026) WEXITSTATUS 1 Nov 05 11:42:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_16(8174026) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_44(8174054) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_44(8174054) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_49(8174059) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_49(8174059) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_52(8174062) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_52(8174062) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_48(8174058) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_48(8174058) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_90(8174100) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_90(8174100) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_91(8174101) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_91(8174101) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_53(8174063) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_53(8174063) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_42(8174052) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_42(8174052) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_21(8174031) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_21(8174031) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_20(8174030) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_20(8174030) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_86(8174096) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_86(8174096) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_17(8174027) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_17(8174027) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_85(8174095) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_85(8174095) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_81(8174091) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_81(8174091) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_89(8174099) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_89(8174099) done Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_40(8174050) WEXITSTATUS 1 Nov 05 11:42:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_40(8174050) done Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_29(8174039) WEXITSTATUS 1 Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_29(8174039) done Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_27(8174037) WEXITSTATUS 1 Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_27(8174037) done Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_28(8174038) WEXITSTATUS 1 Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_28(8174038) done Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_80(8174090) WEXITSTATUS 1 Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_80(8174090) done Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_78(8174088) WEXITSTATUS 1 Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_78(8174088) done Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_79(8174089) WEXITSTATUS 1 Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_79(8174089) done Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_87(8174097) WEXITSTATUS 1 Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_87(8174097) done Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_88(8174098) WEXITSTATUS 1 Nov 05 11:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8173040_88(8174098) done Nov 05 11:42:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4379(8155206) WEXITSTATUS 1 Nov 05 11:42:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4379(8155206) done Nov 05 11:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8149(8172912) WEXITSTATUS 1 Nov 05 11:43:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8149(8172912) done Nov 05 11:43:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174104 InitPrio=23747 usec=4291 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:43:57 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4020(8172954) WEXITSTATUS 0 Nov 05 11:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4020(8172954) done Nov 05 11:44:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7891(8170009) WEXITSTATUS 1 Nov 05 11:44:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7891(8170009) done Nov 05 11:44:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1047(7573216) Nov 05 11:44:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174105 InitPrio=47293 usec=3909 Nov 05 11:44:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7668(8168304) WEXITSTATUS 1 Nov 05 11:44:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7668(8168304) done Nov 05 11:44:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8103(8172744) WEXITSTATUS 1 Nov 05 11:44:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8103(8172744) done Nov 05 11:44:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174106 InitPrio=17501 usec=7237 Nov 05 11:44:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6868(8165091) WEXITSTATUS 1 Nov 05 11:44:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6868(8165091) done Nov 05 11:44:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8168(8174008) WEXITSTATUS 1 Nov 05 11:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8168(8174008) done Nov 05 11:44:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8154(8172959) WEXITSTATUS 1 Nov 05 11:44:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8154(8172959) done Nov 05 11:44:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_1 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_2 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_5 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_6 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_8 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_10 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_4 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_7 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_9 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_3 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_11 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_12 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_13 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_14 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_15 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_16 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_17 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_18 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_19 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_20 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_21 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_22 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_23 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_24 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_25 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_30 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_31 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_28 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_37 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_33 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_34 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_35 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_36 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_26 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_27 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_32 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_29 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_38 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_43 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_44 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_41 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_42 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_39 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_40 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_45 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_50 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_47 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_48 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_57 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_46 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_55 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_56 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_54 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_58 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_52 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_53 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_60 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_49 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_62 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_63 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_65 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_51 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_59 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_68 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_69 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_70 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_71 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_64 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_77 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_73 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_66 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_75 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_76 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_74 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_67 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_72 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_61 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_78 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_87 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_80 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_81 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_82 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_79 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_83 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_84 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_89 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_90 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_88 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_93 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_86 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_91 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_92 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_85 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_2 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_95 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_96 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_5 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173041_94 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_3 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_8 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_9 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_6 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_7 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_12 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_13 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_14 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_15 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_4 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_1 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_10 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_19 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_16 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_25 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_26 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_22 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_23 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_24 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_17 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_11 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_28 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_21 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_29 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_30 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_31 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_32 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_18 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_35 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_20 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_41 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_37 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_34 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_36 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_33 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_42 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_39 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_40 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_38 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_43 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_46 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_27 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_48 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_49 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_47 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_44 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_57 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_45 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_54 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_55 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_56 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_50 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_51 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_52 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_53 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_58 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_59 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_60 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_65 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_62 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_67 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_64 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_61 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_66 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_71 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_68 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_73 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_70 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_75 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_76 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_74 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_72 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_77 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_79 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_80 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_69 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_63 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_81 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_78 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_85 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_86 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_88 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_84 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_82 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_91 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_83 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_89 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_90 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_87 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_92 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_93 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_2 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_95 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_96 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_1 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173137_94 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_7 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_12 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_9 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_10 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_3 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_16 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_8 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_13 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_14 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_11 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_5 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_18 uid 23340 Nov 05 11:44:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_19 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_4 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_25 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_21 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_6 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_23 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_20 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_17 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_26 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_27 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_28 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_22 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_35 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_24 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_29 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_30 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_31 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_32 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_37 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_34 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_39 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_36 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_33 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_38 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_15 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_40 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_41 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_42 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_47 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_44 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_49 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_46 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_51 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_48 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_45 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_50 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_59 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_43 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_56 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_62 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_58 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_60 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_57 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_53 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_54 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_63 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_64 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_65 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_55 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_72 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_69 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_70 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_67 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_68 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_61 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_66 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_71 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_52 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_77 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_74 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_79 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_80 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_73 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_78 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_75 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_76 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_89 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_81 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_86 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_83 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_84 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_82 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_87 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_92 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_85 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_90 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_95 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_88 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_1 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_2 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_91 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_3 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_93 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_6 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_11 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_7 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_8 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_9 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_10 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_12 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_5 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_94 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173233_96 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_16 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_17 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_14 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_15 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_4 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_13 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_22 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_19 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_20 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_21 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_18 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_23 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_24 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_29 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_30 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_35 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_36 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_32 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_33 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_39 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_34 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_27 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_37 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_38 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_31 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_28 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_42 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_26 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_48 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_40 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_45 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_25 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_47 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_41 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_46 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_43 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_52 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_53 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_50 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_51 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_56 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_61 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_54 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_55 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_44 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_49 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_57 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_62 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_59 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_60 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_66 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_63 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_72 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_65 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_58 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_67 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_64 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_68 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_73 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_74 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_75 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_69 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_70 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_71 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_76 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_81 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_78 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_79 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_84 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_85 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_82 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_83 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_80 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_77 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_94 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_90 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_88 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_89 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_86 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_87 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_91 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_92 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_1 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_95 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_96 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_5 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_7 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_8 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_11 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_10 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173329_93 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_3 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_13 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_14 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_2 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_12 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_9 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_4 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_6 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_15 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_16 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_18 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_19 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_21 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_20 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_17 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_22 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_23 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_25 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_26 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_28 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_29 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_31 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_27 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_24 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_35 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_36 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_37 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_38 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_39 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_44 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_45 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_46 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_41 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_42 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_43 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_40 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_47 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_48 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_49 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_50 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_34 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_52 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_32 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_54 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_51 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_56 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_57 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_58 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_59 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_60 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_65 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_61 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_62 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_63 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_64 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_66 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_67 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_68 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_69 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_70 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_55 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_30 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_77 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_53 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_74 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_75 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_72 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_33 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_79 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_76 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_73 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_78 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_71 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_80 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_85 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_82 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_83 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_88 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_89 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_90 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_87 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_84 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_81 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_94 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_91 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_96 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_93 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_86 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_3 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_4 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_92 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_5 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_2 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_7 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_9 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_10 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173425_95 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_16 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_13 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_14 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_11 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_12 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_21 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_17 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_6 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_19 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_20 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_22 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_23 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_24 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_8 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_1 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_18 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_28 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_25 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_30 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_27 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_15 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_33 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_34 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_31 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_32 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_37 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_26 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_35 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_40 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_29 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_42 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_39 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_44 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_45 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_38 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_43 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_36 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_49 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_50 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_47 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_52 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_41 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_54 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_59 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_51 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_56 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_57 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_58 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_60 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_53 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_62 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_55 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_64 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_69 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_61 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_46 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_67 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_68 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_66 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_63 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_72 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_65 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_74 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_71 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_48 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_77 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_70 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_79 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_76 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_85 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_81 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_78 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_83 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_84 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_82 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_75 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_80 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_73 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_90 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_87 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_92 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_89 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_94 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_91 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_93 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_88 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_86 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_95 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_4 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_1 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_2 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_3 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173521_96 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_13 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_9 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_6 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_7 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_12 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_10 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_19 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_16 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_5 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_14 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_11 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_15 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_20 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_21 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_22 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_28 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_24 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_25 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_18 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_27 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_29 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_26 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_23 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_8 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_33 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_30 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_39 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_36 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_37 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_38 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_31 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_35 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_40 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_17 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_42 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_32 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_41 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_46 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_43 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_44 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_53 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_50 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_51 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_48 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_45 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_54 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_47 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_56 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_57 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_58 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_55 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_60 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_61 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_34 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_63 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_64 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_49 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_62 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_59 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_52 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_65 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_74 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_71 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_67 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_72 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_73 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_70 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_76 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_77 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_66 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_83 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_79 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_80 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_81 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_78 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_75 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_85 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_69 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_91 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_88 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_86 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_89 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_90 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_84 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_92 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_94 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_87 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_95 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_96 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_1 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_82 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_4 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_93 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_2 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_3 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173617_68 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_13 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_9 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_6 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_11 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_8 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_10 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_14 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_20 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_16 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_17 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_23 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_7 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_12 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_21 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_27 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_15 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_19 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_25 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_18 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_22 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_33 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_24 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_30 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_31 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_28 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_5 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_26 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_32 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_29 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_34 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_35 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_36 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_37 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_38 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_39 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_44 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_45 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_50 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_46 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_43 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_40 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_41 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_51 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_52 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_49 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_42 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_55 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_48 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_57 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_54 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_59 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_56 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_53 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_62 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_47 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_60 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_65 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_58 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_63 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_68 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_61 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_66 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_75 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_67 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_64 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_73 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_70 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_72 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_69 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_78 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_71 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_76 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_81 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_74 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_79 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_80 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_77 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_82 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_83 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_84 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_85 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_86 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_87 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_92 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_89 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_94 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_3 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_95 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_5 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_93 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_90 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_8 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_96 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_1 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_91 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_4 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173713_88 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_9 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_10 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_16 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_6 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_13 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_2 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_15 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_12 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_14 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_23 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_20 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_17 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_18 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_19 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_21 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_22 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_26 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_11 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_32 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_28 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_34 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_7 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_30 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_27 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_24 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_33 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_35 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_36 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_25 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_29 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_31 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_37 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_42 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_39 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_45 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_43 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_48 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_38 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_41 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_46 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_40 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_44 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_51 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_47 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_49 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_52 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_54 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_50 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_57 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_58 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_64 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_60 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_56 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_53 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_59 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_61 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_62 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_67 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_68 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_65 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_66 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_63 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_55 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_69 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_78 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_74 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_75 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_76 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_82 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_83 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_71 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_80 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_73 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_77 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_88 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_84 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_85 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_91 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_70 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_79 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_89 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_90 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_87 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_93 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_94 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_86 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_96 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_1 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_81 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_95 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_92 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173809_72 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_2 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_7 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_12 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_8 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_5 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_6 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_11 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_9 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_10 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_3 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_16 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_17 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_14 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_15 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_4 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_13 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_18 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_23 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_24 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_21 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_22 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_27 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_28 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_26 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_29 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_19 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_31 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_33 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_34 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_35 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_25 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_37 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_20 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_36 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_39 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_32 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_41 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_43 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_44 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_49 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_46 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_38 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_40 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_48 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_54 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_50 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_56 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_52 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_53 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_55 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_60 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_47 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_62 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_58 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_59 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_61 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_66 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_63 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_68 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_65 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_30 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_45 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_72 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_67 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_57 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_70 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_71 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_64 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_69 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_79 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_75 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_51 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_82 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_73 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_78 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_80 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_86 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_77 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_83 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_42 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_85 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_87 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_84 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_76 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_94 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_81 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_91 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_88 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_93 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_95 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_92 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_89 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_90 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_96 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173905_74 uid 23340 Nov 05 11:44:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8173040_[92-96] uid 23340 Nov 05 11:44:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174107 InitPrio=13783 usec=4019 Nov 05 11:44:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174106_0(8174106) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 11:44:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174103 in day on r807u03n02 Nov 05 11:44:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174108 InitPrio=29089 usec=4116 Nov 05 11:44:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174103 WEXITSTATUS 0 Nov 05 11:44:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174103 done Nov 05 11:44:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:44:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4021(8174109) in gpu on r4513u26n01 Nov 05 11:44:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r914u01n03 Nov 05 11:45:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7972(8170953) WEXITSTATUS 1 Nov 05 11:45:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7972(8170953) done Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8171(8174110) in scavenge on r801u30n03 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8172(8174111) in scavenge on r904u01n02 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8173(8174112) in scavenge on r815u15n12 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8174(8174113) in scavenge on r816u15n11 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8175(8174114) in scavenge on r816u15n12 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8176(8174115) in scavenge on r909u04n04 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8177(8174116) in scavenge on r919u01n03 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8178(8174117) in scavenge on r919u01n03 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8179(8174118) in scavenge on r919u09n01 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8180(8174119) in scavenge on r806u06n04 Nov 05 11:45:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_101(8114367) in scavenge on r815u15n12 Nov 05 11:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2531(8152689) WEXITSTATUS 0 Nov 05 11:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2531(8152689) done Nov 05 11:45:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2963(8174120) in scavenge on r801u04n01 Nov 05 11:45:09 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2964(8174121) in scavenge on r815u23n07 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:45:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:45:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174122 InitPrio=54584 usec=7596 Nov 05 11:45:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174122 NodeList=r807u03n02 #CPUs=1 Partition=day Nov 05 11:45:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174122 WEXITSTATUS 127 Nov 05 11:45:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174122 done Nov 05 11:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174106_0(8174106) WEXITSTATUS 1 Nov 05 11:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174106_0(8174106) done Nov 05 11:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2809(8153316) WEXITSTATUS 0 Nov 05 11:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2809(8153316) done Nov 05 11:45:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135447 WEXITSTATUS 0 Nov 05 11:45:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135447 done Nov 05 11:45:21 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u13n02 Nov 05 11:45:21 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u09n04 Nov 05 11:45:21 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u13n01 Nov 05 11:45:21 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u09n03 Nov 05 11:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8177(8174116) WEXITSTATUS 1 Nov 05 11:45:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8177(8174116) done Nov 05 11:45:52 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8123792_860(8127974) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:45:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8123792_860(8127974) Nov 05 11:46:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174123 InitPrio=20704 usec=5892 Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_0(8174124) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_1(8174125) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_2(8174126) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_3(8174127) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_4(8174128) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_5(8174129) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_6(8174130) NodeList=r915u29n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_7(8174131) NodeList=r915u29n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_8(8174132) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_9(8174133) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_10(8174134) NodeList=r914u09n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_11(8174135) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_12(8174136) NodeList=r914u13n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_13(8174137) NodeList=r914u13n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_14(8174138) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_15(8174139) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:03 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_16(8174140) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:46:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7895(8170102) WEXITSTATUS 1 Nov 05 11:46:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7895(8170102) done Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174141 InitPrio=13477 usec=2545 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174142 InitPrio=13477 usec=2262 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_1(unfulfilled) for JobId=8174142_1(8174143) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_1 uid=23340 usec=167 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_2(unfulfilled) for JobId=8174142_2(8174144) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_2 uid=23340 usec=120 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_3(unfulfilled) for JobId=8174142_3(8174145) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_3 uid=23340 usec=116 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_4(unfulfilled) for JobId=8174142_4(8174146) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_4 uid=23340 usec=115 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_5(unfulfilled) for JobId=8174142_5(8174147) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_5 uid=23340 usec=112 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_6(unfulfilled) for JobId=8174142_6(8174148) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_6 uid=23340 usec=111 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_7(unfulfilled) for JobId=8174142_7(8174149) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_7 uid=23340 usec=109 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_8(unfulfilled) for JobId=8174142_8(8174150) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_8 uid=23340 usec=111 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_9(unfulfilled) for JobId=8174142_9(8174151) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_9 uid=23340 usec=118 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_10(unfulfilled) for JobId=8174142_10(8174152) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_10 uid=23340 usec=111 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_11(unfulfilled) for JobId=8174142_11(8174153) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_11 uid=23340 usec=164 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_12(unfulfilled) for JobId=8174142_12(8174154) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_12 uid=23340 usec=107 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_13(unfulfilled) for JobId=8174142_13(8174155) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_13 uid=23340 usec=111 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_14(unfulfilled) for JobId=8174142_14(8174156) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_14 uid=23340 usec=111 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_15(unfulfilled) for JobId=8174142_15(8174157) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_15 uid=23340 usec=109 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_16(unfulfilled) for JobId=8174142_16(8174158) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_16 uid=23340 usec=112 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_17(unfulfilled) for JobId=8174142_17(8174159) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_17 uid=23340 usec=109 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_18(unfulfilled) for JobId=8174142_18(8174160) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_18 uid=23340 usec=113 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_19(unfulfilled) for JobId=8174142_19(8174161) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_19 uid=23340 usec=109 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_20(unfulfilled) for JobId=8174142_20(8174162) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_20 uid=23340 usec=109 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_21(unfulfilled) for JobId=8174142_21(8174163) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_21 uid=23340 usec=113 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_22(unfulfilled) for JobId=8174142_22(8174164) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_22 uid=23340 usec=107 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_23(unfulfilled) for JobId=8174142_23(8174165) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_23 uid=23340 usec=109 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_24(unfulfilled) for JobId=8174142_24(8174166) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_24 uid=23340 usec=115 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_25(unfulfilled) for JobId=8174142_25(8174167) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_25 uid=23340 usec=292 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_26(unfulfilled) for JobId=8174142_26(8174168) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_26 uid=23340 usec=178 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_27(unfulfilled) for JobId=8174142_27(8174169) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_27 uid=23340 usec=108 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_28(unfulfilled) for JobId=8174142_28(8174170) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_28 uid=23340 usec=108 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_29(unfulfilled) for JobId=8174142_29(8174171) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_29 uid=23340 usec=1954 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_30(unfulfilled) for JobId=8174142_30(8174172) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_30 uid=23340 usec=1108 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_31(unfulfilled) for JobId=8174142_31(8174173) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_31 uid=23340 usec=914 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_32(unfulfilled) for JobId=8174142_32(8174174) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_32 uid=23340 usec=2531 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_33(unfulfilled) for JobId=8174142_33(8174175) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_33 uid=23340 usec=3313 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_34(unfulfilled) for JobId=8174142_34(8174176) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_34 uid=23340 usec=537 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_35(unfulfilled) for JobId=8174142_35(8174177) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_35 uid=23340 usec=115 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_36(unfulfilled) for JobId=8174142_36(8174178) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_36 uid=23340 usec=117 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_37(unfulfilled) for JobId=8174142_37(8174179) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_37 uid=23340 usec=118 Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_38(unfulfilled) for JobId=8174142_38(8174180) Nov 05 11:46:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_38 uid=23340 usec=114 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_39(unfulfilled) for JobId=8174142_39(8174181) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_39 uid=23340 usec=60361 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_40(unfulfilled) for JobId=8174142_40(8174182) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_40 uid=23340 usec=73432 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_41(unfulfilled) for JobId=8174142_41(8174183) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_41 uid=23340 usec=140 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_42(unfulfilled) for JobId=8174142_42(8174184) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_42 uid=23340 usec=122 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_43(unfulfilled) for JobId=8174142_43(8174185) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_43 uid=23340 usec=119 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_44(unfulfilled) for JobId=8174142_44(8174186) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_44 uid=23340 usec=125 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_45(unfulfilled) for JobId=8174142_45(8174187) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_45 uid=23340 usec=113 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_46(unfulfilled) for JobId=8174142_46(8174188) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_46 uid=23340 usec=117 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_47(unfulfilled) for JobId=8174142_47(8174189) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_47 uid=23340 usec=112 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_48(unfulfilled) for JobId=8174142_48(8174190) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_48 uid=23340 usec=121 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_49(unfulfilled) for JobId=8174142_49(8174191) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_49 uid=23340 usec=113 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_50(unfulfilled) for JobId=8174142_50(8174192) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_50 uid=23340 usec=111 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_51(unfulfilled) for JobId=8174142_51(8174193) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_51 uid=23340 usec=110 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_52(unfulfilled) for JobId=8174142_52(8174194) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_52 uid=23340 usec=118 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_53(unfulfilled) for JobId=8174142_53(8174195) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_53 uid=23340 usec=114 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_54(unfulfilled) for JobId=8174142_54(8174196) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_54 uid=23340 usec=110 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_55(unfulfilled) for JobId=8174142_55(8174197) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_55 uid=23340 usec=111 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_56(unfulfilled) for JobId=8174142_56(8174198) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_56 uid=23340 usec=110 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_57(unfulfilled) for JobId=8174142_57(8174199) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_57 uid=23340 usec=142 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_58(unfulfilled) for JobId=8174142_58(8174200) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_58 uid=23340 usec=111 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_59(unfulfilled) for JobId=8174142_59(8174201) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_59 uid=23340 usec=222 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_60(unfulfilled) for JobId=8174142_60(8174202) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_60 uid=23340 usec=149 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_61(unfulfilled) for JobId=8174142_61(8174203) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_61 uid=23340 usec=116 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_62(unfulfilled) for JobId=8174142_62(8174204) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_62 uid=23340 usec=110 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_63(unfulfilled) for JobId=8174142_63(8174205) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_63 uid=23340 usec=111 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_64(unfulfilled) for JobId=8174142_64(8174206) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_64 uid=23340 usec=110 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_65(unfulfilled) for JobId=8174142_65(8174207) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_65 uid=23340 usec=108 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_66(unfulfilled) for JobId=8174142_66(8174208) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_66 uid=23340 usec=178 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_67(unfulfilled) for JobId=8174142_67(8174209) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_67 uid=23340 usec=147 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_68(unfulfilled) for JobId=8174142_68(8174210) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_68 uid=23340 usec=128 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_69(unfulfilled) for JobId=8174142_69(8174211) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_69 uid=23340 usec=133 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_70(unfulfilled) for JobId=8174142_70(8174212) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_70 uid=23340 usec=127 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_71(unfulfilled) for JobId=8174142_71(8174213) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_71 uid=23340 usec=121 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_72(unfulfilled) for JobId=8174142_72(8174214) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_72 uid=23340 usec=114 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_73(unfulfilled) for JobId=8174142_73(8174215) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_73 uid=23340 usec=117 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_74(unfulfilled) for JobId=8174142_74(8174216) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_74 uid=23340 usec=115 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_75(unfulfilled) for JobId=8174142_75(8174217) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_75 uid=23340 usec=116 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_76(unfulfilled) for JobId=8174142_76(8174218) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_76 uid=23340 usec=111 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_77(unfulfilled) for JobId=8174142_77(8174219) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_77 uid=23340 usec=112 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_78(unfulfilled) for JobId=8174142_78(8174220) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_78 uid=23340 usec=122 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_79(unfulfilled) for JobId=8174142_79(8174221) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_79 uid=23340 usec=226 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_80(unfulfilled) for JobId=8174142_80(8174222) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_80 uid=23340 usec=120 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_81(unfulfilled) for JobId=8174142_81(8174223) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_81 uid=23340 usec=304 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_82(unfulfilled) for JobId=8174142_82(8174224) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_82 uid=23340 usec=118 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_83(unfulfilled) for JobId=8174142_83(8174225) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_83 uid=23340 usec=118 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_84(unfulfilled) for JobId=8174142_84(8174226) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_84 uid=23340 usec=146 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_85(unfulfilled) for JobId=8174142_85(8174227) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_85 uid=23340 usec=42108 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_86(unfulfilled) for JobId=8174142_86(8174228) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_86 uid=23340 usec=161 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_87(unfulfilled) for JobId=8174142_87(8174229) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_87 uid=23340 usec=129 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_88(unfulfilled) for JobId=8174142_88(8174230) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_88 uid=23340 usec=129 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_89(unfulfilled) for JobId=8174142_89(8174231) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_89 uid=23340 usec=126 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_90(unfulfilled) for JobId=8174142_90(8174232) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_90 uid=23340 usec=127 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_91(unfulfilled) for JobId=8174142_91(8174233) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_91 uid=23340 usec=122 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_92(unfulfilled) for JobId=8174142_92(8174234) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_92 uid=23340 usec=122 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_93(unfulfilled) for JobId=8174142_93(8174235) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_93 uid=23340 usec=126 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_94(unfulfilled) for JobId=8174142_94(8174236) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_94 uid=23340 usec=129 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_95(unfulfilled) for JobId=8174142_95(8174237) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_95 uid=23340 usec=124 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174141_96(unfulfilled) for JobId=8174142_96(8174142) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174142_96 uid=23340 usec=121 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174238 InitPrio=13477 usec=2441 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_1(unfulfilled) for JobId=8174238_1(8174239) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_1 uid=23340 usec=149 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_2(unfulfilled) for JobId=8174238_2(8174240) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_2 uid=23340 usec=125 Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_3(unfulfilled) for JobId=8174238_3(8174241) Nov 05 11:46:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_3 uid=23340 usec=1117 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_4(unfulfilled) for JobId=8174238_4(8174242) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_4 uid=23340 usec=108375 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_5(unfulfilled) for JobId=8174238_5(8174243) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_5 uid=23340 usec=250 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_6(unfulfilled) for JobId=8174238_6(8174244) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_6 uid=23340 usec=130 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_7(unfulfilled) for JobId=8174238_7(8174245) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_7 uid=23340 usec=126 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_8(unfulfilled) for JobId=8174238_8(8174246) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_8 uid=23340 usec=135 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_9(unfulfilled) for JobId=8174238_9(8174247) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_9 uid=23340 usec=317 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_10(unfulfilled) for JobId=8174238_10(8174248) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_10 uid=23340 usec=284 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_11(unfulfilled) for JobId=8174238_11(8174249) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_11 uid=23340 usec=248 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_12(unfulfilled) for JobId=8174238_12(8174250) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_12 uid=23340 usec=118 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_13(unfulfilled) for JobId=8174238_13(8174251) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_13 uid=23340 usec=116 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_14(unfulfilled) for JobId=8174238_14(8174252) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_14 uid=23340 usec=129 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_15(unfulfilled) for JobId=8174238_15(8174253) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_15 uid=23340 usec=116 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_16(unfulfilled) for JobId=8174238_16(8174254) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_16 uid=23340 usec=127 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_17(unfulfilled) for JobId=8174238_17(8174255) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_17 uid=23340 usec=117 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_18(unfulfilled) for JobId=8174238_18(8174256) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_18 uid=23340 usec=115 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_19(unfulfilled) for JobId=8174238_19(8174257) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_19 uid=23340 usec=113 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_20(unfulfilled) for JobId=8174238_20(8174258) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_20 uid=23340 usec=115 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_21(unfulfilled) for JobId=8174238_21(8174259) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_21 uid=23340 usec=292 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_22(unfulfilled) for JobId=8174238_22(8174260) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_22 uid=23340 usec=113 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_23(unfulfilled) for JobId=8174238_23(8174261) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_23 uid=23340 usec=114 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_24(unfulfilled) for JobId=8174238_24(8174262) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_24 uid=23340 usec=112 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_25(unfulfilled) for JobId=8174238_25(8174263) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_25 uid=23340 usec=114 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_26(unfulfilled) for JobId=8174238_26(8174264) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_26 uid=23340 usec=153 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_27(unfulfilled) for JobId=8174238_27(8174265) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_27 uid=23340 usec=114 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_28(unfulfilled) for JobId=8174238_28(8174266) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_28 uid=23340 usec=112 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_29(unfulfilled) for JobId=8174238_29(8174267) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_29 uid=23340 usec=44862 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_30(unfulfilled) for JobId=8174238_30(8174268) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_30 uid=23340 usec=145 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_31(unfulfilled) for JobId=8174238_31(8174269) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_31 uid=23340 usec=123 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_32(unfulfilled) for JobId=8174238_32(8174270) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_32 uid=23340 usec=117 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_33(unfulfilled) for JobId=8174238_33(8174271) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_33 uid=23340 usec=119 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_34(unfulfilled) for JobId=8174238_34(8174272) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_34 uid=23340 usec=3594 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_35(unfulfilled) for JobId=8174238_35(8174273) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_35 uid=23340 usec=131 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_36(unfulfilled) for JobId=8174238_36(8174274) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_36 uid=23340 usec=127 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_37(unfulfilled) for JobId=8174238_37(8174275) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_37 uid=23340 usec=117 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_38(unfulfilled) for JobId=8174238_38(8174276) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_38 uid=23340 usec=115 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_39(unfulfilled) for JobId=8174238_39(8174277) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_39 uid=23340 usec=114 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_40(unfulfilled) for JobId=8174238_40(8174278) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_40 uid=23340 usec=114 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_41(unfulfilled) for JobId=8174238_41(8174279) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_41 uid=23340 usec=112 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_42(unfulfilled) for JobId=8174238_42(8174280) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_42 uid=23340 usec=115 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_43(unfulfilled) for JobId=8174238_43(8174281) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_43 uid=23340 usec=116 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_44(unfulfilled) for JobId=8174238_44(8174282) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_44 uid=23340 usec=114 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_45(unfulfilled) for JobId=8174238_45(8174283) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_45 uid=23340 usec=115 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_46(unfulfilled) for JobId=8174238_46(8174284) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_46 uid=23340 usec=123 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_47(unfulfilled) for JobId=8174238_47(8174285) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_47 uid=23340 usec=118 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_48(unfulfilled) for JobId=8174238_48(8174286) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_48 uid=23340 usec=116 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_49(unfulfilled) for JobId=8174238_49(8174287) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_49 uid=23340 usec=113 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_50(unfulfilled) for JobId=8174238_50(8174288) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_50 uid=23340 usec=112 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_51(unfulfilled) for JobId=8174238_51(8174289) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_51 uid=23340 usec=116 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_52(unfulfilled) for JobId=8174238_52(8174290) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_52 uid=23340 usec=115 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_53(unfulfilled) for JobId=8174238_53(8174291) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_53 uid=23340 usec=118 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_54(unfulfilled) for JobId=8174238_54(8174292) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_54 uid=23340 usec=118 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_55(unfulfilled) for JobId=8174238_55(8174293) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_55 uid=23340 usec=116 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_56(unfulfilled) for JobId=8174238_56(8174294) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_56 uid=23340 usec=115 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_57(unfulfilled) for JobId=8174238_57(8174295) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_57 uid=23340 usec=117 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_58(unfulfilled) for JobId=8174238_58(8174296) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_58 uid=23340 usec=118 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_59(unfulfilled) for JobId=8174238_59(8174297) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_59 uid=23340 usec=120 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_60(unfulfilled) for JobId=8174238_60(8174298) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_60 uid=23340 usec=116 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_61(unfulfilled) for JobId=8174238_61(8174299) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_61 uid=23340 usec=127 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_62(unfulfilled) for JobId=8174238_62(8174300) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_62 uid=23340 usec=119 Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_63(unfulfilled) for JobId=8174238_63(8174301) Nov 05 11:46:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_63 uid=23340 usec=117 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_64(unfulfilled) for JobId=8174238_64(8174302) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_64 uid=23340 usec=18834 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_65(unfulfilled) for JobId=8174238_65(8174303) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_65 uid=23340 usec=173 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_66(unfulfilled) for JobId=8174238_66(8174304) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_66 uid=23340 usec=123 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_67(unfulfilled) for JobId=8174238_67(8174305) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_67 uid=23340 usec=118 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_68(unfulfilled) for JobId=8174238_68(8174306) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_68 uid=23340 usec=117 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_69(unfulfilled) for JobId=8174238_69(8174307) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_69 uid=23340 usec=121 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_70(unfulfilled) for JobId=8174238_70(8174308) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_70 uid=23340 usec=277 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_71(unfulfilled) for JobId=8174238_71(8174309) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_71 uid=23340 usec=116 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_72(unfulfilled) for JobId=8174238_72(8174310) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_72 uid=23340 usec=130 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_73(unfulfilled) for JobId=8174238_73(8174311) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_73 uid=23340 usec=128 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_74(unfulfilled) for JobId=8174238_74(8174312) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_74 uid=23340 usec=120 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_75(unfulfilled) for JobId=8174238_75(8174313) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_75 uid=23340 usec=177 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_76(unfulfilled) for JobId=8174238_76(8174314) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_76 uid=23340 usec=121 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_77(unfulfilled) for JobId=8174238_77(8174315) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_77 uid=23340 usec=117 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_78(unfulfilled) for JobId=8174238_78(8174316) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_78 uid=23340 usec=46133 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_79(unfulfilled) for JobId=8174238_79(8174317) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_79 uid=23340 usec=150 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_80(unfulfilled) for JobId=8174238_80(8174318) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_80 uid=23340 usec=142 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_81(unfulfilled) for JobId=8174238_81(8174319) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_81 uid=23340 usec=128 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_82(unfulfilled) for JobId=8174238_82(8174320) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_82 uid=23340 usec=122 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_83(unfulfilled) for JobId=8174238_83(8174321) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_83 uid=23340 usec=124 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_84(unfulfilled) for JobId=8174238_84(8174322) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_84 uid=23340 usec=126 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_85(unfulfilled) for JobId=8174238_85(8174323) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_85 uid=23340 usec=116 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_86(unfulfilled) for JobId=8174238_86(8174324) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_86 uid=23340 usec=117 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_87(unfulfilled) for JobId=8174238_87(8174325) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_87 uid=23340 usec=116 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_88(unfulfilled) for JobId=8174238_88(8174326) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_88 uid=23340 usec=115 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_89(unfulfilled) for JobId=8174238_89(8174327) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_89 uid=23340 usec=114 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_90(unfulfilled) for JobId=8174238_90(8174328) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_90 uid=23340 usec=120 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_91(unfulfilled) for JobId=8174238_91(8174329) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_91 uid=23340 usec=115 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_92(unfulfilled) for JobId=8174238_92(8174330) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_92 uid=23340 usec=125 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_93(unfulfilled) for JobId=8174238_93(8174331) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_93 uid=23340 usec=117 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_94(unfulfilled) for JobId=8174238_94(8174332) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_94 uid=23340 usec=113 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_95(unfulfilled) for JobId=8174238_95(8174333) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_95 uid=23340 usec=112 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174142_96(unfulfilled) for JobId=8174238_96(8174238) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174238_96 uid=23340 usec=102 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174334 InitPrio=13477 usec=2439 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_1(unfulfilled) for JobId=8174334_1(8174335) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_1 uid=23340 usec=135 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_2(unfulfilled) for JobId=8174334_2(8174336) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_2 uid=23340 usec=123 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_3(unfulfilled) for JobId=8174334_3(8174337) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_3 uid=23340 usec=274 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_4(unfulfilled) for JobId=8174334_4(8174338) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_4 uid=23340 usec=115 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_5(unfulfilled) for JobId=8174334_5(8174339) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_5 uid=23340 usec=114 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_6(unfulfilled) for JobId=8174334_6(8174340) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_6 uid=23340 usec=117 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_7(unfulfilled) for JobId=8174334_7(8174341) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_7 uid=23340 usec=118 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_8(unfulfilled) for JobId=8174334_8(8174342) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_8 uid=23340 usec=115 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_9(unfulfilled) for JobId=8174334_9(8174343) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_9 uid=23340 usec=120 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_10(unfulfilled) for JobId=8174334_10(8174344) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_10 uid=23340 usec=118 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_11(unfulfilled) for JobId=8174334_11(8174345) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_11 uid=23340 usec=139 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_12(unfulfilled) for JobId=8174334_12(8174346) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_12 uid=23340 usec=176 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_13(unfulfilled) for JobId=8174334_13(8174347) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_13 uid=23340 usec=146 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_14(unfulfilled) for JobId=8174334_14(8174348) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_14 uid=23340 usec=142 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_15(unfulfilled) for JobId=8174334_15(8174349) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_15 uid=23340 usec=129 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_16(unfulfilled) for JobId=8174334_16(8174350) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_16 uid=23340 usec=133 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_17(unfulfilled) for JobId=8174334_17(8174351) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_17 uid=23340 usec=129 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_18(unfulfilled) for JobId=8174334_18(8174352) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_18 uid=23340 usec=298 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_19(unfulfilled) for JobId=8174334_19(8174353) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_19 uid=23340 usec=128 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_20(unfulfilled) for JobId=8174334_20(8174354) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_20 uid=23340 usec=127 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_21(unfulfilled) for JobId=8174334_21(8174355) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_21 uid=23340 usec=125 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_22(unfulfilled) for JobId=8174334_22(8174356) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_22 uid=23340 usec=149 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_23(unfulfilled) for JobId=8174334_23(8174357) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_23 uid=23340 usec=135 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_24(unfulfilled) for JobId=8174334_24(8174358) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_24 uid=23340 usec=128 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_25(unfulfilled) for JobId=8174334_25(8174359) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_25 uid=23340 usec=124 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_26(unfulfilled) for JobId=8174334_26(8174360) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_26 uid=23340 usec=126 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_27(unfulfilled) for JobId=8174334_27(8174361) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_27 uid=23340 usec=123 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_28(unfulfilled) for JobId=8174334_28(8174362) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_28 uid=23340 usec=121 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_29(unfulfilled) for JobId=8174334_29(8174363) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_29 uid=23340 usec=181 Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_30(unfulfilled) for JobId=8174334_30(8174364) Nov 05 11:46:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_30 uid=23340 usec=151 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_31(unfulfilled) for JobId=8174334_31(8174365) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_31 uid=23340 usec=61222 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_32(unfulfilled) for JobId=8174334_32(8174366) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_32 uid=23340 usec=207 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_33(unfulfilled) for JobId=8174334_33(8174367) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_33 uid=23340 usec=130 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_34(unfulfilled) for JobId=8174334_34(8174368) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_34 uid=23340 usec=126 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_35(unfulfilled) for JobId=8174334_35(8174369) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_35 uid=23340 usec=122 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_36(unfulfilled) for JobId=8174334_36(8174370) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_36 uid=23340 usec=121 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_37(unfulfilled) for JobId=8174334_37(8174371) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_37 uid=23340 usec=121 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_38(unfulfilled) for JobId=8174334_38(8174372) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_38 uid=23340 usec=122 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_39(unfulfilled) for JobId=8174334_39(8174373) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_39 uid=23340 usec=121 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_40(unfulfilled) for JobId=8174334_40(8174374) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_40 uid=23340 usec=130 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_41(unfulfilled) for JobId=8174334_41(8174375) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_41 uid=23340 usec=123 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_42(unfulfilled) for JobId=8174334_42(8174376) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_42 uid=23340 usec=122 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_43(unfulfilled) for JobId=8174334_43(8174377) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_43 uid=23340 usec=119 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_44(unfulfilled) for JobId=8174334_44(8174378) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_44 uid=23340 usec=119 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_45(unfulfilled) for JobId=8174334_45(8174379) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_45 uid=23340 usec=129 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_46(unfulfilled) for JobId=8174334_46(8174380) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_46 uid=23340 usec=127 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_47(unfulfilled) for JobId=8174334_47(8174381) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_47 uid=23340 usec=126 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_48(unfulfilled) for JobId=8174334_48(8174382) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_48 uid=23340 usec=123 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_49(unfulfilled) for JobId=8174334_49(8174383) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_49 uid=23340 usec=118 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_50(unfulfilled) for JobId=8174334_50(8174384) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_50 uid=23340 usec=174 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_51(unfulfilled) for JobId=8174334_51(8174385) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_51 uid=23340 usec=122 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_52(unfulfilled) for JobId=8174334_52(8174386) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_52 uid=23340 usec=118 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_53(unfulfilled) for JobId=8174334_53(8174387) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_53 uid=23340 usec=275 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_54(unfulfilled) for JobId=8174334_54(8174388) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_54 uid=23340 usec=123 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_55(unfulfilled) for JobId=8174334_55(8174389) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_55 uid=23340 usec=125 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_56(unfulfilled) for JobId=8174334_56(8174390) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_56 uid=23340 usec=230 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_57(unfulfilled) for JobId=8174334_57(8174391) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_57 uid=23340 usec=298 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_58(unfulfilled) for JobId=8174334_58(8174392) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_58 uid=23340 usec=124 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_59(unfulfilled) for JobId=8174334_59(8174393) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_59 uid=23340 usec=120 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_60(unfulfilled) for JobId=8174334_60(8174394) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_60 uid=23340 usec=120 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_61(unfulfilled) for JobId=8174334_61(8174395) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_61 uid=23340 usec=138 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_62(unfulfilled) for JobId=8174334_62(8174396) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_62 uid=23340 usec=134 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_63(unfulfilled) for JobId=8174334_63(8174397) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_63 uid=23340 usec=124 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_64(unfulfilled) for JobId=8174334_64(8174398) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_64 uid=23340 usec=125 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_65(unfulfilled) for JobId=8174334_65(8174399) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_65 uid=23340 usec=115 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_66(unfulfilled) for JobId=8174334_66(8174400) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_66 uid=23340 usec=118 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_67(unfulfilled) for JobId=8174334_67(8174401) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_67 uid=23340 usec=117 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_68(unfulfilled) for JobId=8174334_68(8174402) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_68 uid=23340 usec=122 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_69(unfulfilled) for JobId=8174334_69(8174403) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_69 uid=23340 usec=124 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_70(unfulfilled) for JobId=8174334_70(8174404) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_70 uid=23340 usec=119 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_71(unfulfilled) for JobId=8174334_71(8174405) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_71 uid=23340 usec=122 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_72(unfulfilled) for JobId=8174334_72(8174406) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_72 uid=23340 usec=118 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_73(unfulfilled) for JobId=8174334_73(8174407) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_73 uid=23340 usec=118 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_74(unfulfilled) for JobId=8174334_74(8174408) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_74 uid=23340 usec=116 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_75(unfulfilled) for JobId=8174334_75(8174409) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_75 uid=23340 usec=284 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_76(unfulfilled) for JobId=8174334_76(8174410) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_76 uid=23340 usec=118 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_77(unfulfilled) for JobId=8174334_77(8174411) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_77 uid=23340 usec=305 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_78(unfulfilled) for JobId=8174334_78(8174412) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_78 uid=23340 usec=138 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_79(unfulfilled) for JobId=8174334_79(8174413) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_79 uid=23340 usec=119 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_80(unfulfilled) for JobId=8174334_80(8174414) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_80 uid=23340 usec=35274 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_81(unfulfilled) for JobId=8174334_81(8174415) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_81 uid=23340 usec=148 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_82(unfulfilled) for JobId=8174334_82(8174416) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_82 uid=23340 usec=131 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_83(unfulfilled) for JobId=8174334_83(8174417) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_83 uid=23340 usec=130 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_84(unfulfilled) for JobId=8174334_84(8174418) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_84 uid=23340 usec=126 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_85(unfulfilled) for JobId=8174334_85(8174419) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_85 uid=23340 usec=126 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_86(unfulfilled) for JobId=8174334_86(8174420) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_86 uid=23340 usec=121 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_87(unfulfilled) for JobId=8174334_87(8174421) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_87 uid=23340 usec=130 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_88(unfulfilled) for JobId=8174334_88(8174422) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_88 uid=23340 usec=119 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_89(unfulfilled) for JobId=8174334_89(8174423) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_89 uid=23340 usec=118 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_90(unfulfilled) for JobId=8174334_90(8174424) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_90 uid=23340 usec=117 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_91(unfulfilled) for JobId=8174334_91(8174425) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_91 uid=23340 usec=122 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_92(unfulfilled) for JobId=8174334_92(8174426) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_92 uid=23340 usec=132 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_93(unfulfilled) for JobId=8174334_93(8174427) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_93 uid=23340 usec=116 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_94(unfulfilled) for JobId=8174334_94(8174428) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_94 uid=23340 usec=116 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_95(unfulfilled) for JobId=8174334_95(8174429) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_95 uid=23340 usec=134 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174238_96(unfulfilled) for JobId=8174334_96(8174334) Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174334_96 uid=23340 usec=108 Nov 05 11:46:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174430 InitPrio=13477 usec=2440 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_1(unfulfilled) for JobId=8174430_1(8174431) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_1 uid=23340 usec=93701 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_2(unfulfilled) for JobId=8174430_2(8174432) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_2 uid=23340 usec=190 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_3(unfulfilled) for JobId=8174430_3(8174433) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_3 uid=23340 usec=136 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_4(unfulfilled) for JobId=8174430_4(8174434) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_4 uid=23340 usec=121 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_5(unfulfilled) for JobId=8174430_5(8174435) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_5 uid=23340 usec=124 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_6(unfulfilled) for JobId=8174430_6(8174436) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_6 uid=23340 usec=121 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_7(unfulfilled) for JobId=8174430_7(8174437) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_7 uid=23340 usec=119 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_8(unfulfilled) for JobId=8174430_8(8174438) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_8 uid=23340 usec=118 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_9(unfulfilled) for JobId=8174430_9(8174439) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_9 uid=23340 usec=121 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_10(unfulfilled) for JobId=8174430_10(8174440) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_10 uid=23340 usec=117 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_11(unfulfilled) for JobId=8174430_11(8174441) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_11 uid=23340 usec=118 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_12(unfulfilled) for JobId=8174430_12(8174442) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_12 uid=23340 usec=174 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_13(unfulfilled) for JobId=8174430_13(8174443) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_13 uid=23340 usec=264 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_14(unfulfilled) for JobId=8174430_14(8174444) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_14 uid=23340 usec=116 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_15(unfulfilled) for JobId=8174430_15(8174445) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_15 uid=23340 usec=113 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_16(unfulfilled) for JobId=8174430_16(8174446) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_16 uid=23340 usec=140 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_17(unfulfilled) for JobId=8174430_17(8174447) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_17 uid=23340 usec=119 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_18(unfulfilled) for JobId=8174430_18(8174448) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_18 uid=23340 usec=115 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_19(unfulfilled) for JobId=8174430_19(8174449) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_19 uid=23340 usec=132 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_20(unfulfilled) for JobId=8174430_20(8174450) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_20 uid=23340 usec=114 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_21(unfulfilled) for JobId=8174430_21(8174451) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_21 uid=23340 usec=112 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_22(unfulfilled) for JobId=8174430_22(8174452) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_22 uid=23340 usec=175 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_23(unfulfilled) for JobId=8174430_23(8174453) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_23 uid=23340 usec=138 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_24(unfulfilled) for JobId=8174430_24(8174454) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_24 uid=23340 usec=144 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_25(unfulfilled) for JobId=8174430_25(8174455) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_25 uid=23340 usec=140 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_26(unfulfilled) for JobId=8174430_26(8174456) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_26 uid=23340 usec=124 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_27(unfulfilled) for JobId=8174430_27(8174457) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_27 uid=23340 usec=180 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_28(unfulfilled) for JobId=8174430_28(8174458) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_28 uid=23340 usec=48176 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_29(unfulfilled) for JobId=8174430_29(8174459) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_29 uid=23340 usec=156 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_30(unfulfilled) for JobId=8174430_30(8174460) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_30 uid=23340 usec=124 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_31(unfulfilled) for JobId=8174430_31(8174461) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_31 uid=23340 usec=114 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_32(unfulfilled) for JobId=8174430_32(8174462) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_32 uid=23340 usec=114 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_33(unfulfilled) for JobId=8174430_33(8174463) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_33 uid=23340 usec=111 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_34(unfulfilled) for JobId=8174430_34(8174464) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_34 uid=23340 usec=113 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_35(unfulfilled) for JobId=8174430_35(8174465) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_35 uid=23340 usec=145 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_36(unfulfilled) for JobId=8174430_36(8174466) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_36 uid=23340 usec=126 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_37(unfulfilled) for JobId=8174430_37(8174467) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_37 uid=23340 usec=120 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_38(unfulfilled) for JobId=8174430_38(8174468) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_38 uid=23340 usec=112 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_39(unfulfilled) for JobId=8174430_39(8174469) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_39 uid=23340 usec=109 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_40(unfulfilled) for JobId=8174430_40(8174470) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_40 uid=23340 usec=107 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_41(unfulfilled) for JobId=8174430_41(8174471) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_41 uid=23340 usec=115 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_42(unfulfilled) for JobId=8174430_42(8174472) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_42 uid=23340 usec=112 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_43(unfulfilled) for JobId=8174430_43(8174473) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_43 uid=23340 usec=115 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_44(unfulfilled) for JobId=8174430_44(8174474) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_44 uid=23340 usec=120 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_45(unfulfilled) for JobId=8174430_45(8174475) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_45 uid=23340 usec=136 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_46(unfulfilled) for JobId=8174430_46(8174476) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_46 uid=23340 usec=124 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_47(unfulfilled) for JobId=8174430_47(8174477) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_47 uid=23340 usec=115 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_48(unfulfilled) for JobId=8174430_48(8174478) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_48 uid=23340 usec=111 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_49(unfulfilled) for JobId=8174430_49(8174479) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_49 uid=23340 usec=118 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_50(unfulfilled) for JobId=8174430_50(8174480) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_50 uid=23340 usec=112 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_51(unfulfilled) for JobId=8174430_51(8174481) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_51 uid=23340 usec=282 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_52(unfulfilled) for JobId=8174430_52(8174482) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_52 uid=23340 usec=119 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_53(unfulfilled) for JobId=8174430_53(8174483) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_53 uid=23340 usec=112 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_54(unfulfilled) for JobId=8174430_54(8174484) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_54 uid=23340 usec=111 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_55(unfulfilled) for JobId=8174430_55(8174485) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_55 uid=23340 usec=113 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_56(unfulfilled) for JobId=8174430_56(8174486) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_56 uid=23340 usec=126 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_57(unfulfilled) for JobId=8174430_57(8174487) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_57 uid=23340 usec=115 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_58(unfulfilled) for JobId=8174430_58(8174488) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_58 uid=23340 usec=112 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_59(unfulfilled) for JobId=8174430_59(8174489) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_59 uid=23340 usec=123 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_60(unfulfilled) for JobId=8174430_60(8174490) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_60 uid=23340 usec=117 Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_61(unfulfilled) for JobId=8174430_61(8174491) Nov 05 11:46:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_61 uid=23340 usec=127 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_62(unfulfilled) for JobId=8174430_62(8174492) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_62 uid=23340 usec=296 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_63(unfulfilled) for JobId=8174430_63(8174493) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_63 uid=23340 usec=18737 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_64(unfulfilled) for JobId=8174430_64(8174494) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_64 uid=23340 usec=240 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_65(unfulfilled) for JobId=8174430_65(8174495) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_65 uid=23340 usec=128 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_66(unfulfilled) for JobId=8174430_66(8174496) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_66 uid=23340 usec=114 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_67(unfulfilled) for JobId=8174430_67(8174497) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_67 uid=23340 usec=110 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_68(unfulfilled) for JobId=8174430_68(8174498) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_68 uid=23340 usec=112 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_69(unfulfilled) for JobId=8174430_69(8174499) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_69 uid=23340 usec=115 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_70(unfulfilled) for JobId=8174430_70(8174500) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_70 uid=23340 usec=112 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_71(unfulfilled) for JobId=8174430_71(8174501) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_71 uid=23340 usec=111 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_72(unfulfilled) for JobId=8174430_72(8174502) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_72 uid=23340 usec=302 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_73(unfulfilled) for JobId=8174430_73(8174503) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_73 uid=23340 usec=300 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_74(unfulfilled) for JobId=8174430_74(8174504) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_74 uid=23340 usec=109 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_75(unfulfilled) for JobId=8174430_75(8174505) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_75 uid=23340 usec=165 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_76(unfulfilled) for JobId=8174430_76(8174506) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_76 uid=23340 usec=140 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_77(unfulfilled) for JobId=8174430_77(8174507) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_77 uid=23340 usec=309 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_78(unfulfilled) for JobId=8174430_78(8174508) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_78 uid=23340 usec=47281 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_79(unfulfilled) for JobId=8174430_79(8174509) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_79 uid=23340 usec=163 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_80(unfulfilled) for JobId=8174430_80(8174510) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_80 uid=23340 usec=142 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_81(unfulfilled) for JobId=8174430_81(8174511) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_81 uid=23340 usec=128 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_82(unfulfilled) for JobId=8174430_82(8174512) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_82 uid=23340 usec=136 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_83(unfulfilled) for JobId=8174430_83(8174513) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_83 uid=23340 usec=128 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_84(unfulfilled) for JobId=8174430_84(8174514) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_84 uid=23340 usec=128 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_85(unfulfilled) for JobId=8174430_85(8174515) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_85 uid=23340 usec=135 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_86(unfulfilled) for JobId=8174430_86(8174516) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_86 uid=23340 usec=127 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_87(unfulfilled) for JobId=8174430_87(8174517) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_87 uid=23340 usec=131 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_88(unfulfilled) for JobId=8174430_88(8174518) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_88 uid=23340 usec=125 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_89(unfulfilled) for JobId=8174430_89(8174519) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_89 uid=23340 usec=124 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_90(unfulfilled) for JobId=8174430_90(8174520) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_90 uid=23340 usec=132 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_91(unfulfilled) for JobId=8174430_91(8174521) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_91 uid=23340 usec=131 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_92(unfulfilled) for JobId=8174430_92(8174522) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_92 uid=23340 usec=130 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_93(unfulfilled) for JobId=8174430_93(8174523) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_93 uid=23340 usec=131 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_94(unfulfilled) for JobId=8174430_94(8174524) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_94 uid=23340 usec=129 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_95(unfulfilled) for JobId=8174430_95(8174525) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_95 uid=23340 usec=126 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174334_96(unfulfilled) for JobId=8174430_96(8174430) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174430_96 uid=23340 usec=121 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174526 InitPrio=13477 usec=2505 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_1(unfulfilled) for JobId=8174526_1(8174527) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_1 uid=23340 usec=131 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_2(unfulfilled) for JobId=8174526_2(8174528) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_2 uid=23340 usec=177 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_3(unfulfilled) for JobId=8174526_3(8174529) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_3 uid=23340 usec=126 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_4(unfulfilled) for JobId=8174526_4(8174530) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_4 uid=23340 usec=123 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_5(unfulfilled) for JobId=8174526_5(8174531) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_5 uid=23340 usec=132 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_6(unfulfilled) for JobId=8174526_6(8174532) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_6 uid=23340 usec=127 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_7(unfulfilled) for JobId=8174526_7(8174533) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_7 uid=23340 usec=128 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_8(unfulfilled) for JobId=8174526_8(8174534) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_8 uid=23340 usec=312 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_9(unfulfilled) for JobId=8174526_9(8174535) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_9 uid=23340 usec=122 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_10(unfulfilled) for JobId=8174526_10(8174536) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_10 uid=23340 usec=120 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_11(unfulfilled) for JobId=8174526_11(8174537) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_11 uid=23340 usec=247 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_12(unfulfilled) for JobId=8174526_12(8174538) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_12 uid=23340 usec=296 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_13(unfulfilled) for JobId=8174526_13(8174539) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_13 uid=23340 usec=114 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_14(unfulfilled) for JobId=8174526_14(8174540) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_14 uid=23340 usec=130 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_15(unfulfilled) for JobId=8174526_15(8174541) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_15 uid=23340 usec=122 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_16(unfulfilled) for JobId=8174526_16(8174542) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_16 uid=23340 usec=128 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_17(unfulfilled) for JobId=8174526_17(8174543) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_17 uid=23340 usec=128 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_18(unfulfilled) for JobId=8174526_18(8174544) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_18 uid=23340 usec=127 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_19(unfulfilled) for JobId=8174526_19(8174545) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_19 uid=23340 usec=132 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_20(unfulfilled) for JobId=8174526_20(8174546) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_20 uid=23340 usec=287 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_21(unfulfilled) for JobId=8174526_21(8174547) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_21 uid=23340 usec=130 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_22(unfulfilled) for JobId=8174526_22(8174548) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_22 uid=23340 usec=119 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_23(unfulfilled) for JobId=8174526_23(8174549) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_23 uid=23340 usec=127 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_24(unfulfilled) for JobId=8174526_24(8174550) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_24 uid=23340 usec=121 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_25(unfulfilled) for JobId=8174526_25(8174551) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_25 uid=23340 usec=121 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_26(unfulfilled) for JobId=8174526_26(8174552) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_26 uid=23340 usec=121 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_27(unfulfilled) for JobId=8174526_27(8174553) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_27 uid=23340 usec=117 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_28(unfulfilled) for JobId=8174526_28(8174554) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_28 uid=23340 usec=244 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_29(unfulfilled) for JobId=8174526_29(8174555) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_29 uid=23340 usec=119 Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_30(unfulfilled) for JobId=8174526_30(8174556) Nov 05 11:46:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_30 uid=23340 usec=193 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_31(unfulfilled) for JobId=8174526_31(8174557) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_31 uid=23340 usec=158653 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_32(unfulfilled) for JobId=8174526_32(8174558) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_32 uid=23340 usec=76081 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_33(unfulfilled) for JobId=8174526_33(8174559) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_33 uid=23340 usec=149 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_34(unfulfilled) for JobId=8174526_34(8174560) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_34 uid=23340 usec=160 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_35(unfulfilled) for JobId=8174526_35(8174561) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_35 uid=23340 usec=133 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_36(unfulfilled) for JobId=8174526_36(8174562) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_36 uid=23340 usec=119 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_37(unfulfilled) for JobId=8174526_37(8174563) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_37 uid=23340 usec=126 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_38(unfulfilled) for JobId=8174526_38(8174564) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_38 uid=23340 usec=114 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_39(unfulfilled) for JobId=8174526_39(8174565) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_39 uid=23340 usec=119 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_40(unfulfilled) for JobId=8174526_40(8174566) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_40 uid=23340 usec=115 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_41(unfulfilled) for JobId=8174526_41(8174567) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_41 uid=23340 usec=118 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_42(unfulfilled) for JobId=8174526_42(8174568) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_42 uid=23340 usec=115 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_43(unfulfilled) for JobId=8174526_43(8174569) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_43 uid=23340 usec=117 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_44(unfulfilled) for JobId=8174526_44(8174570) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_44 uid=23340 usec=140 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_45(unfulfilled) for JobId=8174526_45(8174571) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_45 uid=23340 usec=129 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_46(unfulfilled) for JobId=8174526_46(8174572) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_46 uid=23340 usec=115 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_47(unfulfilled) for JobId=8174526_47(8174573) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_47 uid=23340 usec=115 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_48(unfulfilled) for JobId=8174526_48(8174574) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_48 uid=23340 usec=115 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_49(unfulfilled) for JobId=8174526_49(8174575) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_49 uid=23340 usec=114 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_50(unfulfilled) for JobId=8174526_50(8174576) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_50 uid=23340 usec=115 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_51(unfulfilled) for JobId=8174526_51(8174577) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_51 uid=23340 usec=116 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_52(unfulfilled) for JobId=8174526_52(8174578) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_52 uid=23340 usec=113 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_53(unfulfilled) for JobId=8174526_53(8174579) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_53 uid=23340 usec=115 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_54(unfulfilled) for JobId=8174526_54(8174580) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_54 uid=23340 usec=277 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_55(unfulfilled) for JobId=8174526_55(8174581) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_55 uid=23340 usec=116 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_56(unfulfilled) for JobId=8174526_56(8174582) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_56 uid=23340 usec=111 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_57(unfulfilled) for JobId=8174526_57(8174583) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_57 uid=23340 usec=114 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_58(unfulfilled) for JobId=8174526_58(8174584) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_58 uid=23340 usec=119 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_59(unfulfilled) for JobId=8174526_59(8174585) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_59 uid=23340 usec=112 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_60(unfulfilled) for JobId=8174526_60(8174586) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_60 uid=23340 usec=5461 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_61(unfulfilled) for JobId=8174526_61(8174587) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_61 uid=23340 usec=143 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_62(unfulfilled) for JobId=8174526_62(8174588) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_62 uid=23340 usec=124 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_63(unfulfilled) for JobId=8174526_63(8174589) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_63 uid=23340 usec=134 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_64(unfulfilled) for JobId=8174526_64(8174590) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_64 uid=23340 usec=180 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_65(unfulfilled) for JobId=8174526_65(8174591) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_65 uid=23340 usec=306 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_66(unfulfilled) for JobId=8174526_66(8174592) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_66 uid=23340 usec=121 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_67(unfulfilled) for JobId=8174526_67(8174593) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_67 uid=23340 usec=40449 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_68(unfulfilled) for JobId=8174526_68(8174594) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_68 uid=23340 usec=154 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_69(unfulfilled) for JobId=8174526_69(8174595) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_69 uid=23340 usec=127 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_70(unfulfilled) for JobId=8174526_70(8174596) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_70 uid=23340 usec=123 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_71(unfulfilled) for JobId=8174526_71(8174597) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_71 uid=23340 usec=118 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_72(unfulfilled) for JobId=8174526_72(8174598) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_72 uid=23340 usec=146 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_73(unfulfilled) for JobId=8174526_73(8174599) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_73 uid=23340 usec=131 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_74(unfulfilled) for JobId=8174526_74(8174600) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_74 uid=23340 usec=126 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_75(unfulfilled) for JobId=8174526_75(8174601) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_75 uid=23340 usec=136 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_76(unfulfilled) for JobId=8174526_76(8174602) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_76 uid=23340 usec=128 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_77(unfulfilled) for JobId=8174526_77(8174603) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_77 uid=23340 usec=118 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_78(unfulfilled) for JobId=8174526_78(8174604) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_78 uid=23340 usec=116 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_79(unfulfilled) for JobId=8174526_79(8174605) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_79 uid=23340 usec=147 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_80(unfulfilled) for JobId=8174526_80(8174606) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_80 uid=23340 usec=134 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_81(unfulfilled) for JobId=8174526_81(8174607) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_81 uid=23340 usec=116 Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_82(unfulfilled) for JobId=8174526_82(8174608) Nov 05 11:46:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_82 uid=23340 usec=115 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_83(unfulfilled) for JobId=8174526_83(8174609) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_83 uid=23340 usec=13155 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_84(unfulfilled) for JobId=8174526_84(8174610) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_84 uid=23340 usec=117 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_85(unfulfilled) for JobId=8174526_85(8174611) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_85 uid=23340 usec=118 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_86(unfulfilled) for JobId=8174526_86(8174612) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_86 uid=23340 usec=303 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_87(unfulfilled) for JobId=8174526_87(8174613) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_87 uid=23340 usec=156 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_88(unfulfilled) for JobId=8174526_88(8174614) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_88 uid=23340 usec=273 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_89(unfulfilled) for JobId=8174526_89(8174615) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_89 uid=23340 usec=161 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_90(unfulfilled) for JobId=8174526_90(8174616) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_90 uid=23340 usec=117 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_91(unfulfilled) for JobId=8174526_91(8174617) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_91 uid=23340 usec=213 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_92(unfulfilled) for JobId=8174526_92(8174618) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_92 uid=23340 usec=108 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_93(unfulfilled) for JobId=8174526_93(8174619) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_93 uid=23340 usec=110 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_94(unfulfilled) for JobId=8174526_94(8174620) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_94 uid=23340 usec=154 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_95(unfulfilled) for JobId=8174526_95(8174621) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_95 uid=23340 usec=113 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174430_96(unfulfilled) for JobId=8174526_96(8174526) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174526_96 uid=23340 usec=99 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174622 InitPrio=13477 usec=2267 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_1(unfulfilled) for JobId=8174622_1(8174623) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_1 uid=23340 usec=128 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_2(unfulfilled) for JobId=8174622_2(8174624) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_2 uid=23340 usec=123 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_3(unfulfilled) for JobId=8174622_3(8174625) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_3 uid=23340 usec=112 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_4(unfulfilled) for JobId=8174622_4(8174626) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_4 uid=23340 usec=110 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_5(unfulfilled) for JobId=8174622_5(8174627) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_5 uid=23340 usec=107 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_6(unfulfilled) for JobId=8174622_6(8174628) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_6 uid=23340 usec=113 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_7(unfulfilled) for JobId=8174622_7(8174629) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_7 uid=23340 usec=109 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_8(unfulfilled) for JobId=8174622_8(8174630) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_8 uid=23340 usec=109 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_9(unfulfilled) for JobId=8174622_9(8174631) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_9 uid=23340 usec=109 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_10(unfulfilled) for JobId=8174622_10(8174632) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_10 uid=23340 usec=107 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_11(unfulfilled) for JobId=8174622_11(8174633) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_11 uid=23340 usec=109 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_12(unfulfilled) for JobId=8174622_12(8174634) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_12 uid=23340 usec=172 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_13(unfulfilled) for JobId=8174622_13(8174635) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_13 uid=23340 usec=112 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_14(unfulfilled) for JobId=8174622_14(8174636) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_14 uid=23340 usec=107 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_15(unfulfilled) for JobId=8174622_15(8174637) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_15 uid=23340 usec=107 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_16(unfulfilled) for JobId=8174622_16(8174638) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_16 uid=23340 usec=183 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_17(unfulfilled) for JobId=8174622_17(8174639) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_17 uid=23340 usec=108 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_18(unfulfilled) for JobId=8174622_18(8174640) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_18 uid=23340 usec=143 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_19(unfulfilled) for JobId=8174622_19(8174641) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_19 uid=23340 usec=35516 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_20(unfulfilled) for JobId=8174622_20(8174642) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_20 uid=23340 usec=158 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_21(unfulfilled) for JobId=8174622_21(8174643) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_21 uid=23340 usec=127 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_22(unfulfilled) for JobId=8174622_22(8174644) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_22 uid=23340 usec=117 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_23(unfulfilled) for JobId=8174622_23(8174645) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_23 uid=23340 usec=117 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_24(unfulfilled) for JobId=8174622_24(8174646) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_24 uid=23340 usec=141 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_25(unfulfilled) for JobId=8174622_25(8174647) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_25 uid=23340 usec=119 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_26(unfulfilled) for JobId=8174622_26(8174648) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_26 uid=23340 usec=124 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_27(unfulfilled) for JobId=8174622_27(8174649) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_27 uid=23340 usec=124 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_28(unfulfilled) for JobId=8174622_28(8174650) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_28 uid=23340 usec=127 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_29(unfulfilled) for JobId=8174622_29(8174651) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_29 uid=23340 usec=117 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_30(unfulfilled) for JobId=8174622_30(8174652) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_30 uid=23340 usec=114 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_31(unfulfilled) for JobId=8174622_31(8174653) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_31 uid=23340 usec=115 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_32(unfulfilled) for JobId=8174622_32(8174654) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_32 uid=23340 usec=113 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_33(unfulfilled) for JobId=8174622_33(8174655) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_33 uid=23340 usec=134 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_34(unfulfilled) for JobId=8174622_34(8174656) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_34 uid=23340 usec=128 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_35(unfulfilled) for JobId=8174622_35(8174657) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_35 uid=23340 usec=123 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_36(unfulfilled) for JobId=8174622_36(8174658) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_36 uid=23340 usec=130 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_37(unfulfilled) for JobId=8174622_37(8174659) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_37 uid=23340 usec=116 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_38(unfulfilled) for JobId=8174622_38(8174660) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_38 uid=23340 usec=113 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_39(unfulfilled) for JobId=8174622_39(8174661) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_39 uid=23340 usec=115 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_40(unfulfilled) for JobId=8174622_40(8174662) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_40 uid=23340 usec=115 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_41(unfulfilled) for JobId=8174622_41(8174663) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_41 uid=23340 usec=114 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_42(unfulfilled) for JobId=8174622_42(8174664) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_42 uid=23340 usec=117 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_43(unfulfilled) for JobId=8174622_43(8174665) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_43 uid=23340 usec=114 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_44(unfulfilled) for JobId=8174622_44(8174666) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_44 uid=23340 usec=116 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_45(unfulfilled) for JobId=8174622_45(8174667) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_45 uid=23340 usec=146 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_46(unfulfilled) for JobId=8174622_46(8174668) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_46 uid=23340 usec=116 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_47(unfulfilled) for JobId=8174622_47(8174669) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_47 uid=23340 usec=114 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_48(unfulfilled) for JobId=8174622_48(8174670) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_48 uid=23340 usec=113 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_49(unfulfilled) for JobId=8174622_49(8174671) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_49 uid=23340 usec=293 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_50(unfulfilled) for JobId=8174622_50(8174672) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_50 uid=23340 usec=114 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_51(unfulfilled) for JobId=8174622_51(8174673) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_51 uid=23340 usec=126 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_52(unfulfilled) for JobId=8174622_52(8174674) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_52 uid=23340 usec=117 Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_53(unfulfilled) for JobId=8174622_53(8174675) Nov 05 11:46:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_53 uid=23340 usec=144 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_54(unfulfilled) for JobId=8174622_54(8174676) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_54 uid=23340 usec=21892 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_55(unfulfilled) for JobId=8174622_55(8174677) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_55 uid=23340 usec=167 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_56(unfulfilled) for JobId=8174622_56(8174678) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_56 uid=23340 usec=128 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_57(unfulfilled) for JobId=8174622_57(8174679) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_57 uid=23340 usec=117 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_58(unfulfilled) for JobId=8174622_58(8174680) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_58 uid=23340 usec=124 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_59(unfulfilled) for JobId=8174622_59(8174681) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_59 uid=23340 usec=118 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_60(unfulfilled) for JobId=8174622_60(8174682) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_60 uid=23340 usec=180 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_61(unfulfilled) for JobId=8174622_61(8174683) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_61 uid=23340 usec=117 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_62(unfulfilled) for JobId=8174622_62(8174684) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_62 uid=23340 usec=114 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_63(unfulfilled) for JobId=8174622_63(8174685) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_63 uid=23340 usec=117 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_64(unfulfilled) for JobId=8174622_64(8174686) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_64 uid=23340 usec=164 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_65(unfulfilled) for JobId=8174622_65(8174687) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_65 uid=23340 usec=117 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_66(unfulfilled) for JobId=8174622_66(8174688) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_66 uid=23340 usec=308 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_67(unfulfilled) for JobId=8174622_67(8174689) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_67 uid=23340 usec=178 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_68(unfulfilled) for JobId=8174622_68(8174690) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_68 uid=23340 usec=125 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_69(unfulfilled) for JobId=8174622_69(8174691) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_69 uid=23340 usec=44725 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_70(unfulfilled) for JobId=8174622_70(8174692) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_70 uid=23340 usec=151 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_71(unfulfilled) for JobId=8174622_71(8174693) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_71 uid=23340 usec=122 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_72(unfulfilled) for JobId=8174622_72(8174694) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_72 uid=23340 usec=127 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_73(unfulfilled) for JobId=8174622_73(8174695) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_73 uid=23340 usec=122 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_74(unfulfilled) for JobId=8174622_74(8174696) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_74 uid=23340 usec=123 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_75(unfulfilled) for JobId=8174622_75(8174697) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_75 uid=23340 usec=126 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_76(unfulfilled) for JobId=8174622_76(8174698) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_76 uid=23340 usec=124 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_77(unfulfilled) for JobId=8174622_77(8174699) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_77 uid=23340 usec=290 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_78(unfulfilled) for JobId=8174622_78(8174700) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_78 uid=23340 usec=121 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_79(unfulfilled) for JobId=8174622_79(8174701) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_79 uid=23340 usec=119 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_80(unfulfilled) for JobId=8174622_80(8174702) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_80 uid=23340 usec=119 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_81(unfulfilled) for JobId=8174622_81(8174703) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_81 uid=23340 usec=128 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_82(unfulfilled) for JobId=8174622_82(8174704) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_82 uid=23340 usec=127 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_83(unfulfilled) for JobId=8174622_83(8174705) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_83 uid=23340 usec=124 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_84(unfulfilled) for JobId=8174622_84(8174706) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_84 uid=23340 usec=120 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_85(unfulfilled) for JobId=8174622_85(8174707) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_85 uid=23340 usec=118 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_86(unfulfilled) for JobId=8174622_86(8174708) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_86 uid=23340 usec=122 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_87(unfulfilled) for JobId=8174622_87(8174709) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_87 uid=23340 usec=121 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_88(unfulfilled) for JobId=8174622_88(8174710) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_88 uid=23340 usec=120 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_89(unfulfilled) for JobId=8174622_89(8174711) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_89 uid=23340 usec=126 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_90(unfulfilled) for JobId=8174622_90(8174712) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_90 uid=23340 usec=124 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_91(unfulfilled) for JobId=8174622_91(8174713) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_91 uid=23340 usec=125 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_92(unfulfilled) for JobId=8174622_92(8174714) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_92 uid=23340 usec=126 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_93(unfulfilled) for JobId=8174622_93(8174715) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_93 uid=23340 usec=166 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_94(unfulfilled) for JobId=8174622_94(8174716) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_94 uid=23340 usec=120 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_95(unfulfilled) for JobId=8174622_95(8174717) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_95 uid=23340 usec=116 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174526_96(unfulfilled) for JobId=8174622_96(8174622) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174622_96 uid=23340 usec=186 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174718 InitPrio=13477 usec=2477 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_1(unfulfilled) for JobId=8174718_1(8174719) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_1 uid=23340 usec=129 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_2(unfulfilled) for JobId=8174718_2(8174720) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_2 uid=23340 usec=117 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7757(8168911) WEXITSTATUS 1 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7757(8168911) done Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_3(unfulfilled) for JobId=8174718_3(8174721) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_3 uid=23340 usec=417 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_4(unfulfilled) for JobId=8174718_4(8174722) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_4 uid=23340 usec=131 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_5(unfulfilled) for JobId=8174718_5(8174723) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_5 uid=23340 usec=125 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_6(unfulfilled) for JobId=8174718_6(8174724) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_6 uid=23340 usec=149 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_7(unfulfilled) for JobId=8174718_7(8174725) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_7 uid=23340 usec=127 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_8(unfulfilled) for JobId=8174718_8(8174726) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_8 uid=23340 usec=119 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_9(unfulfilled) for JobId=8174718_9(8174727) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_9 uid=23340 usec=147 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_10(unfulfilled) for JobId=8174718_10(8174728) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_10 uid=23340 usec=132 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_11(unfulfilled) for JobId=8174718_11(8174729) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_11 uid=23340 usec=142 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_12(unfulfilled) for JobId=8174718_12(8174730) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_12 uid=23340 usec=132 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_13(unfulfilled) for JobId=8174718_13(8174731) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_13 uid=23340 usec=130 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_14(unfulfilled) for JobId=8174718_14(8174732) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_14 uid=23340 usec=126 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_15(unfulfilled) for JobId=8174718_15(8174733) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_15 uid=23340 usec=129 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_16(unfulfilled) for JobId=8174718_16(8174734) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_16 uid=23340 usec=158 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_17(unfulfilled) for JobId=8174718_17(8174735) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_17 uid=23340 usec=142 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_18(unfulfilled) for JobId=8174718_18(8174736) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_18 uid=23340 usec=134 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_19(unfulfilled) for JobId=8174718_19(8174737) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_19 uid=23340 usec=129 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_20(unfulfilled) for JobId=8174718_20(8174738) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_20 uid=23340 usec=139 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_21(unfulfilled) for JobId=8174718_21(8174739) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_21 uid=23340 usec=130 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_22(unfulfilled) for JobId=8174718_22(8174740) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_22 uid=23340 usec=297 Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_23(unfulfilled) for JobId=8174718_23(8174741) Nov 05 11:46:20 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_23 uid=23340 usec=140 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_24(unfulfilled) for JobId=8174718_24(8174742) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_24 uid=23340 usec=54793 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_25(unfulfilled) for JobId=8174718_25(8174743) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_25 uid=23340 usec=158 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_26(unfulfilled) for JobId=8174718_26(8174744) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_26 uid=23340 usec=133 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_27(unfulfilled) for JobId=8174718_27(8174745) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_27 uid=23340 usec=134 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_28(unfulfilled) for JobId=8174718_28(8174746) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_28 uid=23340 usec=133 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_29(unfulfilled) for JobId=8174718_29(8174747) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_29 uid=23340 usec=132 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_30(unfulfilled) for JobId=8174718_30(8174748) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_30 uid=23340 usec=129 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_31(unfulfilled) for JobId=8174718_31(8174749) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_31 uid=23340 usec=130 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_32(unfulfilled) for JobId=8174718_32(8174750) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_32 uid=23340 usec=129 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_33(unfulfilled) for JobId=8174718_33(8174751) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_33 uid=23340 usec=126 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_34(unfulfilled) for JobId=8174718_34(8174752) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_34 uid=23340 usec=128 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_35(unfulfilled) for JobId=8174718_35(8174753) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_35 uid=23340 usec=126 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_36(unfulfilled) for JobId=8174718_36(8174754) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_36 uid=23340 usec=119 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_37(unfulfilled) for JobId=8174718_37(8174755) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_37 uid=23340 usec=127 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_38(unfulfilled) for JobId=8174718_38(8174756) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_38 uid=23340 usec=132 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_39(unfulfilled) for JobId=8174718_39(8174757) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_39 uid=23340 usec=131 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_40(unfulfilled) for JobId=8174718_40(8174758) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_40 uid=23340 usec=119 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_41(unfulfilled) for JobId=8174718_41(8174759) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_41 uid=23340 usec=121 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_42(unfulfilled) for JobId=8174718_42(8174760) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_42 uid=23340 usec=121 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_43(unfulfilled) for JobId=8174718_43(8174761) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_43 uid=23340 usec=133 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_44(unfulfilled) for JobId=8174718_44(8174762) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_44 uid=23340 usec=123 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_45(unfulfilled) for JobId=8174718_45(8174763) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_45 uid=23340 usec=128 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_46(unfulfilled) for JobId=8174718_46(8174764) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_46 uid=23340 usec=125 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_47(unfulfilled) for JobId=8174718_47(8174765) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_47 uid=23340 usec=120 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_48(unfulfilled) for JobId=8174718_48(8174766) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_48 uid=23340 usec=130 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_49(unfulfilled) for JobId=8174718_49(8174767) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_49 uid=23340 usec=123 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_50(unfulfilled) for JobId=8174718_50(8174768) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_50 uid=23340 usec=209 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_51(unfulfilled) for JobId=8174718_51(8174769) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_51 uid=23340 usec=122 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_52(unfulfilled) for JobId=8174718_52(8174770) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_52 uid=23340 usec=121 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_53(unfulfilled) for JobId=8174718_53(8174771) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_53 uid=23340 usec=123 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_54(unfulfilled) for JobId=8174718_54(8174772) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_54 uid=23340 usec=127 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_55(unfulfilled) for JobId=8174718_55(8174773) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_55 uid=23340 usec=124 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_56(unfulfilled) for JobId=8174718_56(8174774) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_56 uid=23340 usec=267 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_57(unfulfilled) for JobId=8174718_57(8174775) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_57 uid=23340 usec=255 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_58(unfulfilled) for JobId=8174718_58(8174776) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_58 uid=23340 usec=122 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_59(unfulfilled) for JobId=8174718_59(8174777) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_59 uid=23340 usec=219 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_60(unfulfilled) for JobId=8174718_60(8174778) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_60 uid=23340 usec=1346 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_61(unfulfilled) for JobId=8174718_61(8174779) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_61 uid=23340 usec=1358 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_62(unfulfilled) for JobId=8174718_62(8174780) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_62 uid=23340 usec=1366 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_63(unfulfilled) for JobId=8174718_63(8174781) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_63 uid=23340 usec=3436 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_64(unfulfilled) for JobId=8174718_64(8174782) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_64 uid=23340 usec=2613 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_65(unfulfilled) for JobId=8174718_65(8174783) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_65 uid=23340 usec=135 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_66(unfulfilled) for JobId=8174718_66(8174784) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_66 uid=23340 usec=136 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_67(unfulfilled) for JobId=8174718_67(8174785) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_67 uid=23340 usec=130 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_68(unfulfilled) for JobId=8174718_68(8174786) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_68 uid=23340 usec=128 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_69(unfulfilled) for JobId=8174718_69(8174787) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_69 uid=23340 usec=314 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_70(unfulfilled) for JobId=8174718_70(8174788) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_70 uid=23340 usec=127 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_71(unfulfilled) for JobId=8174718_71(8174789) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_71 uid=23340 usec=157 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_72(unfulfilled) for JobId=8174718_72(8174790) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_72 uid=23340 usec=126 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_73(unfulfilled) for JobId=8174718_73(8174791) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_73 uid=23340 usec=144 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_74(unfulfilled) for JobId=8174718_74(8174792) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_74 uid=23340 usec=137 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_75(unfulfilled) for JobId=8174718_75(8174793) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_75 uid=23340 usec=37406 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_76(unfulfilled) for JobId=8174718_76(8174794) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_76 uid=23340 usec=159 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_77(unfulfilled) for JobId=8174718_77(8174795) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_77 uid=23340 usec=136 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_78(unfulfilled) for JobId=8174718_78(8174796) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_78 uid=23340 usec=138 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_79(unfulfilled) for JobId=8174718_79(8174797) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_79 uid=23340 usec=129 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_80(unfulfilled) for JobId=8174718_80(8174798) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_80 uid=23340 usec=153 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_81(unfulfilled) for JobId=8174718_81(8174799) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_81 uid=23340 usec=138 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_82(unfulfilled) for JobId=8174718_82(8174800) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_82 uid=23340 usec=127 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_83(unfulfilled) for JobId=8174718_83(8174801) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_83 uid=23340 usec=138 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_84(unfulfilled) for JobId=8174718_84(8174802) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_84 uid=23340 usec=132 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_85(unfulfilled) for JobId=8174718_85(8174803) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_85 uid=23340 usec=139 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_86(unfulfilled) for JobId=8174718_86(8174804) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_86 uid=23340 usec=118 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_87(unfulfilled) for JobId=8174718_87(8174805) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_87 uid=23340 usec=114 Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_88(unfulfilled) for JobId=8174718_88(8174806) Nov 05 11:46:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_88 uid=23340 usec=118 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_89(unfulfilled) for JobId=8174718_89(8174807) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_89 uid=23340 usec=89986 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_90(unfulfilled) for JobId=8174718_90(8174808) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_90 uid=23340 usec=197 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_91(unfulfilled) for JobId=8174718_91(8174809) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_91 uid=23340 usec=159 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_92(unfulfilled) for JobId=8174718_92(8174810) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_92 uid=23340 usec=128 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_93(unfulfilled) for JobId=8174718_93(8174811) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_93 uid=23340 usec=124 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_94(unfulfilled) for JobId=8174718_94(8174812) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_94 uid=23340 usec=122 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_95(unfulfilled) for JobId=8174718_95(8174813) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_95 uid=23340 usec=188 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174622_96(unfulfilled) for JobId=8174718_96(8174718) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174718_96 uid=23340 usec=269 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174814 InitPrio=13477 usec=1866 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_1(unfulfilled) for JobId=8174814_1(8174815) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_1 uid=23340 usec=137 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_2(unfulfilled) for JobId=8174814_2(8174816) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_2 uid=23340 usec=118 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_3(unfulfilled) for JobId=8174814_3(8174817) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_3 uid=23340 usec=123 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_4(unfulfilled) for JobId=8174814_4(8174818) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_4 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_5(unfulfilled) for JobId=8174814_5(8174819) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_5 uid=23340 usec=297 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_6(unfulfilled) for JobId=8174814_6(8174820) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_6 uid=23340 usec=112 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_7(unfulfilled) for JobId=8174814_7(8174821) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_7 uid=23340 usec=116 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_8(unfulfilled) for JobId=8174814_8(8174822) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_8 uid=23340 usec=111 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_9(unfulfilled) for JobId=8174814_9(8174823) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_9 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_10(unfulfilled) for JobId=8174814_10(8174824) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_10 uid=23340 usec=123 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_11(unfulfilled) for JobId=8174814_11(8174825) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_11 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_12(unfulfilled) for JobId=8174814_12(8174826) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_12 uid=23340 usec=189 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_13(unfulfilled) for JobId=8174814_13(8174827) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_13 uid=23340 usec=110 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_14(unfulfilled) for JobId=8174814_14(8174828) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_14 uid=23340 usec=109 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_15(unfulfilled) for JobId=8174814_15(8174829) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_15 uid=23340 usec=111 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_16(unfulfilled) for JobId=8174814_16(8174830) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_16 uid=23340 usec=109 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_17(unfulfilled) for JobId=8174814_17(8174831) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_17 uid=23340 usec=121 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_18(unfulfilled) for JobId=8174814_18(8174832) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_18 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_19(unfulfilled) for JobId=8174814_19(8174833) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_19 uid=23340 usec=276 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_20(unfulfilled) for JobId=8174814_20(8174834) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_20 uid=23340 usec=108 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_21(unfulfilled) for JobId=8174814_21(8174835) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_21 uid=23340 usec=109 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_22(unfulfilled) for JobId=8174814_22(8174836) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_22 uid=23340 usec=40751 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_23(unfulfilled) for JobId=8174814_23(8174837) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_23 uid=23340 usec=153 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_24(unfulfilled) for JobId=8174814_24(8174838) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_24 uid=23340 usec=120 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_25(unfulfilled) for JobId=8174814_25(8174839) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_25 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_26(unfulfilled) for JobId=8174814_26(8174840) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_26 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_27(unfulfilled) for JobId=8174814_27(8174841) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_27 uid=23340 usec=112 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_28(unfulfilled) for JobId=8174814_28(8174842) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_28 uid=23340 usec=125 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_29(unfulfilled) for JobId=8174814_29(8174843) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_29 uid=23340 usec=115 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_30(unfulfilled) for JobId=8174814_30(8174844) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_30 uid=23340 usec=123 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_31(unfulfilled) for JobId=8174814_31(8174845) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_31 uid=23340 usec=117 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_32(unfulfilled) for JobId=8174814_32(8174846) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_32 uid=23340 usec=118 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_33(unfulfilled) for JobId=8174814_33(8174847) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_33 uid=23340 usec=191 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_34(unfulfilled) for JobId=8174814_34(8174848) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_34 uid=23340 usec=119 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_35(unfulfilled) for JobId=8174814_35(8174849) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_35 uid=23340 usec=113 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_36(unfulfilled) for JobId=8174814_36(8174850) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_36 uid=23340 usec=121 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_37(unfulfilled) for JobId=8174814_37(8174851) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_37 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_38(unfulfilled) for JobId=8174814_38(8174852) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_38 uid=23340 usec=112 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_39(unfulfilled) for JobId=8174814_39(8174853) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_39 uid=23340 usec=113 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_40(unfulfilled) for JobId=8174814_40(8174854) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_40 uid=23340 usec=111 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_41(unfulfilled) for JobId=8174814_41(8174855) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_41 uid=23340 usec=113 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_42(unfulfilled) for JobId=8174814_42(8174856) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_42 uid=23340 usec=112 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_43(unfulfilled) for JobId=8174814_43(8174857) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_43 uid=23340 usec=200 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_44(unfulfilled) for JobId=8174814_44(8174858) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_44 uid=23340 usec=118 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_45(unfulfilled) for JobId=8174814_45(8174859) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_45 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_46(unfulfilled) for JobId=8174814_46(8174860) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_46 uid=23340 usec=114 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_47(unfulfilled) for JobId=8174814_47(8174861) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_47 uid=23340 usec=112 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_48(unfulfilled) for JobId=8174814_48(8174862) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_48 uid=23340 usec=113 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_49(unfulfilled) for JobId=8174814_49(8174863) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_49 uid=23340 usec=1135 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_50(unfulfilled) for JobId=8174814_50(8174864) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_50 uid=23340 usec=1257 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_51(unfulfilled) for JobId=8174814_51(8174865) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_51 uid=23340 usec=4332 Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_52(unfulfilled) for JobId=8174814_52(8174866) Nov 05 11:46:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_52 uid=23340 usec=965 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_53(unfulfilled) for JobId=8174814_53(8174867) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_53 uid=23340 usec=104535 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_54(unfulfilled) for JobId=8174814_54(8174868) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_54 uid=23340 usec=248 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_55(unfulfilled) for JobId=8174814_55(8174869) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_55 uid=23340 usec=146 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_56(unfulfilled) for JobId=8174814_56(8174870) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_56 uid=23340 usec=132 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_57(unfulfilled) for JobId=8174814_57(8174871) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_57 uid=23340 usec=167 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_58(unfulfilled) for JobId=8174814_58(8174872) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_58 uid=23340 usec=133 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_59(unfulfilled) for JobId=8174814_59(8174873) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_59 uid=23340 usec=308 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_60(unfulfilled) for JobId=8174814_60(8174874) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_60 uid=23340 usec=282 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_61(unfulfilled) for JobId=8174814_61(8174875) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_61 uid=23340 usec=287 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_62(unfulfilled) for JobId=8174814_62(8174876) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_62 uid=23340 usec=201 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_63(unfulfilled) for JobId=8174814_63(8174877) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_63 uid=23340 usec=45268 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_64(unfulfilled) for JobId=8174814_64(8174878) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_64 uid=23340 usec=175 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_65(unfulfilled) for JobId=8174814_65(8174879) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_65 uid=23340 usec=138 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_66(unfulfilled) for JobId=8174814_66(8174880) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_66 uid=23340 usec=130 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_67(unfulfilled) for JobId=8174814_67(8174881) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_67 uid=23340 usec=118 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_68(unfulfilled) for JobId=8174814_68(8174882) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_68 uid=23340 usec=137 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_69(unfulfilled) for JobId=8174814_69(8174883) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_69 uid=23340 usec=126 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_70(unfulfilled) for JobId=8174814_70(8174884) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_70 uid=23340 usec=117 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_71(unfulfilled) for JobId=8174814_71(8174885) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_71 uid=23340 usec=112 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_72(unfulfilled) for JobId=8174814_72(8174886) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_72 uid=23340 usec=114 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_73(unfulfilled) for JobId=8174814_73(8174887) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_73 uid=23340 usec=111 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_74(unfulfilled) for JobId=8174814_74(8174888) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_74 uid=23340 usec=115 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_75(unfulfilled) for JobId=8174814_75(8174889) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_75 uid=23340 usec=139 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_76(unfulfilled) for JobId=8174814_76(8174890) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_76 uid=23340 usec=124 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_77(unfulfilled) for JobId=8174814_77(8174891) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_77 uid=23340 usec=113 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_78(unfulfilled) for JobId=8174814_78(8174892) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_78 uid=23340 usec=132 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_79(unfulfilled) for JobId=8174814_79(8174893) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_79 uid=23340 usec=121 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_80(unfulfilled) for JobId=8174814_80(8174894) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_80 uid=23340 usec=114 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_81(unfulfilled) for JobId=8174814_81(8174895) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_81 uid=23340 usec=128 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_82(unfulfilled) for JobId=8174814_82(8174896) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_82 uid=23340 usec=133 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_83(unfulfilled) for JobId=8174814_83(8174897) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_83 uid=23340 usec=164 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_84(unfulfilled) for JobId=8174814_84(8174898) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_84 uid=23340 usec=129 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_85(unfulfilled) for JobId=8174814_85(8174899) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_85 uid=23340 usec=134 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_86(unfulfilled) for JobId=8174814_86(8174900) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_86 uid=23340 usec=134 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_87(unfulfilled) for JobId=8174814_87(8174901) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_87 uid=23340 usec=124 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_88(unfulfilled) for JobId=8174814_88(8174902) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_88 uid=23340 usec=130 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_89(unfulfilled) for JobId=8174814_89(8174903) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_89 uid=23340 usec=130 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_90(unfulfilled) for JobId=8174814_90(8174904) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_90 uid=23340 usec=128 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_91(unfulfilled) for JobId=8174814_91(8174905) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_91 uid=23340 usec=177 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_92(unfulfilled) for JobId=8174814_92(8174906) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_92 uid=23340 usec=117 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_93(unfulfilled) for JobId=8174814_93(8174907) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_93 uid=23340 usec=123 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_94(unfulfilled) for JobId=8174814_94(8174908) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_94 uid=23340 usec=294 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_95(unfulfilled) for JobId=8174814_95(8174909) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_95 uid=23340 usec=116 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174718_96(unfulfilled) for JobId=8174814_96(8174814) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174814_96 uid=23340 usec=171 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8174910 InitPrio=13477 usec=2580 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_1(unfulfilled) for JobId=8174910_1(8174911) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_1 uid=23340 usec=166 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_2(unfulfilled) for JobId=8174910_2(8174912) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_2 uid=23340 usec=299 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_3(unfulfilled) for JobId=8174910_3(8174913) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_3 uid=23340 usec=128 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_4(unfulfilled) for JobId=8174910_4(8174914) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_4 uid=23340 usec=118 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_5(unfulfilled) for JobId=8174910_5(8174915) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_5 uid=23340 usec=282 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_6(unfulfilled) for JobId=8174910_6(8174916) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_6 uid=23340 usec=122 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_7(unfulfilled) for JobId=8174910_7(8174917) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_7 uid=23340 usec=149 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_8(unfulfilled) for JobId=8174910_8(8174918) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_8 uid=23340 usec=116 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_9(unfulfilled) for JobId=8174910_9(8174919) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_9 uid=23340 usec=174 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_10(unfulfilled) for JobId=8174910_10(8174920) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_10 uid=23340 usec=118 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_11(unfulfilled) for JobId=8174910_11(8174921) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_11 uid=23340 usec=117 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_12(unfulfilled) for JobId=8174910_12(8174922) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_12 uid=23340 usec=115 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_13(unfulfilled) for JobId=8174910_13(8174923) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_13 uid=23340 usec=127 Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_14(unfulfilled) for JobId=8174910_14(8174924) Nov 05 11:46:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_14 uid=23340 usec=133 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_15(unfulfilled) for JobId=8174910_15(8174925) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_15 uid=23340 usec=63715 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_16(unfulfilled) for JobId=8174910_16(8174926) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_16 uid=23340 usec=233 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_17(unfulfilled) for JobId=8174910_17(8174927) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_17 uid=23340 usec=126 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_18(unfulfilled) for JobId=8174910_18(8174928) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_18 uid=23340 usec=124 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_19(unfulfilled) for JobId=8174910_19(8174929) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_19 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_20(unfulfilled) for JobId=8174910_20(8174930) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_20 uid=23340 usec=120 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_21(unfulfilled) for JobId=8174910_21(8174931) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_21 uid=23340 usec=120 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_22(unfulfilled) for JobId=8174910_22(8174932) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_22 uid=23340 usec=128 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_23(unfulfilled) for JobId=8174910_23(8174933) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_23 uid=23340 usec=125 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_24(unfulfilled) for JobId=8174910_24(8174934) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_24 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_25(unfulfilled) for JobId=8174910_25(8174935) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_25 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_26(unfulfilled) for JobId=8174910_26(8174936) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_26 uid=23340 usec=119 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_27(unfulfilled) for JobId=8174910_27(8174937) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_27 uid=23340 usec=124 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_28(unfulfilled) for JobId=8174910_28(8174938) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_28 uid=23340 usec=121 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_29(unfulfilled) for JobId=8174910_29(8174939) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_29 uid=23340 usec=120 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_30(unfulfilled) for JobId=8174910_30(8174940) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_30 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_31(unfulfilled) for JobId=8174910_31(8174941) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_31 uid=23340 usec=118 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_32(unfulfilled) for JobId=8174910_32(8174942) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_32 uid=23340 usec=116 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_33(unfulfilled) for JobId=8174910_33(8174943) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_33 uid=23340 usec=118 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_34(unfulfilled) for JobId=8174910_34(8174944) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_34 uid=23340 usec=119 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_35(unfulfilled) for JobId=8174910_35(8174945) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_35 uid=23340 usec=116 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_36(unfulfilled) for JobId=8174910_36(8174946) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_36 uid=23340 usec=139 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_37(unfulfilled) for JobId=8174910_37(8174947) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_37 uid=23340 usec=131 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_38(unfulfilled) for JobId=8174910_38(8174948) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_38 uid=23340 usec=126 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_39(unfulfilled) for JobId=8174910_39(8174949) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_39 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_40(unfulfilled) for JobId=8174910_40(8174950) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_40 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_41(unfulfilled) for JobId=8174910_41(8174951) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_41 uid=23340 usec=123 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_42(unfulfilled) for JobId=8174910_42(8174952) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_42 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_43(unfulfilled) for JobId=8174910_43(8174953) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_43 uid=23340 usec=121 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_44(unfulfilled) for JobId=8174910_44(8174954) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_44 uid=23340 usec=140 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_45(unfulfilled) for JobId=8174910_45(8174955) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_45 uid=23340 usec=126 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_46(unfulfilled) for JobId=8174910_46(8174956) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_46 uid=23340 usec=134 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_47(unfulfilled) for JobId=8174910_47(8174957) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_47 uid=23340 usec=140 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_48(unfulfilled) for JobId=8174910_48(8174958) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_48 uid=23340 usec=135 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_49(unfulfilled) for JobId=8174910_49(8174959) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_49 uid=23340 usec=127 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_50(unfulfilled) for JobId=8174910_50(8174960) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_50 uid=23340 usec=118 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_51(unfulfilled) for JobId=8174910_51(8174961) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_51 uid=23340 usec=172 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_52(unfulfilled) for JobId=8174910_52(8174962) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_52 uid=23340 usec=130 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_53(unfulfilled) for JobId=8174910_53(8174963) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_53 uid=23340 usec=123 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_54(unfulfilled) for JobId=8174910_54(8174964) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_54 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_55(unfulfilled) for JobId=8174910_55(8174965) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_55 uid=23340 usec=194 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_56(unfulfilled) for JobId=8174910_56(8174966) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_56 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_57(unfulfilled) for JobId=8174910_57(8174967) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_57 uid=23340 usec=132 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_58(unfulfilled) for JobId=8174910_58(8174968) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_58 uid=23340 usec=133 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_59(unfulfilled) for JobId=8174910_59(8174969) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_59 uid=23340 usec=144 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_60(unfulfilled) for JobId=8174910_60(8174970) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_60 uid=23340 usec=121 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_61(unfulfilled) for JobId=8174910_61(8174971) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_61 uid=23340 usec=118 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_62(unfulfilled) for JobId=8174910_62(8174972) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_62 uid=23340 usec=120 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_63(unfulfilled) for JobId=8174910_63(8174973) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_63 uid=23340 usec=270 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_64(unfulfilled) for JobId=8174910_64(8174974) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_64 uid=23340 usec=157 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_65(unfulfilled) for JobId=8174910_65(8174975) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_65 uid=23340 usec=118 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_66(unfulfilled) for JobId=8174910_66(8174976) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_66 uid=23340 usec=38716 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_67(unfulfilled) for JobId=8174910_67(8174977) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_67 uid=23340 usec=172 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_68(unfulfilled) for JobId=8174910_68(8174978) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_68 uid=23340 usec=124 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_69(unfulfilled) for JobId=8174910_69(8174979) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_69 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_70(unfulfilled) for JobId=8174910_70(8174980) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_70 uid=23340 usec=121 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_71(unfulfilled) for JobId=8174910_71(8174981) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_71 uid=23340 usec=122 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_72(unfulfilled) for JobId=8174910_72(8174982) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_72 uid=23340 usec=120 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_73(unfulfilled) for JobId=8174910_73(8174983) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_73 uid=23340 usec=118 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_74(unfulfilled) for JobId=8174910_74(8174984) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_74 uid=23340 usec=132 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_75(unfulfilled) for JobId=8174910_75(8174985) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_75 uid=23340 usec=126 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_76(unfulfilled) for JobId=8174910_76(8174986) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_76 uid=23340 usec=132 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_77(unfulfilled) for JobId=8174910_77(8174987) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_77 uid=23340 usec=159 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_78(unfulfilled) for JobId=8174910_78(8174988) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_78 uid=23340 usec=149 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_79(unfulfilled) for JobId=8174910_79(8174989) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_79 uid=23340 usec=150 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_80(unfulfilled) for JobId=8174910_80(8174990) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_80 uid=23340 usec=137 Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_81(unfulfilled) for JobId=8174910_81(8174991) Nov 05 11:46:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_81 uid=23340 usec=132 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_82(unfulfilled) for JobId=8174910_82(8174992) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_82 uid=23340 usec=94888 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_83(unfulfilled) for JobId=8174910_83(8174993) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_83 uid=23340 usec=216 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_84(unfulfilled) for JobId=8174910_84(8174994) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_84 uid=23340 usec=124 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_85(unfulfilled) for JobId=8174910_85(8174995) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_85 uid=23340 usec=159 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_86(unfulfilled) for JobId=8174910_86(8174996) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_86 uid=23340 usec=137 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_87(unfulfilled) for JobId=8174910_87(8174997) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_87 uid=23340 usec=121 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_88(unfulfilled) for JobId=8174910_88(8174998) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_88 uid=23340 usec=136 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_89(unfulfilled) for JobId=8174910_89(8174999) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_89 uid=23340 usec=125 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_90(unfulfilled) for JobId=8174910_90(8175000) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_90 uid=23340 usec=118 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_91(unfulfilled) for JobId=8174910_91(8175001) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_91 uid=23340 usec=114 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_92(unfulfilled) for JobId=8174910_92(8175002) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_92 uid=23340 usec=163 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_93(unfulfilled) for JobId=8174910_93(8175003) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_93 uid=23340 usec=305 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_94(unfulfilled) for JobId=8174910_94(8175004) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_94 uid=23340 usec=116 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_95(unfulfilled) for JobId=8174910_95(8175005) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_95 uid=23340 usec=138 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174814_96(unfulfilled) for JobId=8174910_96(8174910) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8174910_96 uid=23340 usec=111 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175006 InitPrio=13477 usec=2835 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_1(unfulfilled) for JobId=8175006_1(8175007) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_1 uid=23340 usec=135 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_2(unfulfilled) for JobId=8175006_2(8175008) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_2 uid=23340 usec=127 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_3(unfulfilled) for JobId=8175006_3(8175009) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_3 uid=23340 usec=116 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_4(unfulfilled) for JobId=8175006_4(8175010) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_4 uid=23340 usec=113 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_5(unfulfilled) for JobId=8175006_5(8175011) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_5 uid=23340 usec=139 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_6(unfulfilled) for JobId=8175006_6(8175012) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_6 uid=23340 usec=126 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_7(unfulfilled) for JobId=8175006_7(8175013) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_7 uid=23340 usec=180 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_8(unfulfilled) for JobId=8175006_8(8175014) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_8 uid=23340 usec=133 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_9(unfulfilled) for JobId=8175006_9(8175015) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_9 uid=23340 usec=124 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_10(unfulfilled) for JobId=8175006_10(8175016) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_10 uid=23340 usec=127 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_11(unfulfilled) for JobId=8175006_11(8175017) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_11 uid=23340 usec=113 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_12(unfulfilled) for JobId=8175006_12(8175018) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_12 uid=23340 usec=45707 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_13(unfulfilled) for JobId=8175006_13(8175019) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_13 uid=23340 usec=160 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_14(unfulfilled) for JobId=8175006_14(8175020) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_14 uid=23340 usec=134 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_15(unfulfilled) for JobId=8175006_15(8175021) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_15 uid=23340 usec=146 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_16(unfulfilled) for JobId=8175006_16(8175022) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_16 uid=23340 usec=132 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_17(unfulfilled) for JobId=8175006_17(8175023) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_17 uid=23340 usec=120 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_18(unfulfilled) for JobId=8175006_18(8175024) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_18 uid=23340 usec=138 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_19(unfulfilled) for JobId=8175006_19(8175025) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_19 uid=23340 usec=129 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_20(unfulfilled) for JobId=8175006_20(8175026) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_20 uid=23340 usec=118 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_21(unfulfilled) for JobId=8175006_21(8175027) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_21 uid=23340 usec=148 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_22(unfulfilled) for JobId=8175006_22(8175028) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_22 uid=23340 usec=131 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_23(unfulfilled) for JobId=8175006_23(8175029) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_23 uid=23340 usec=118 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_24(unfulfilled) for JobId=8175006_24(8175030) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_24 uid=23340 usec=152 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_25(unfulfilled) for JobId=8175006_25(8175031) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_25 uid=23340 usec=129 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_26(unfulfilled) for JobId=8175006_26(8175032) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_26 uid=23340 usec=126 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_27(unfulfilled) for JobId=8175006_27(8175033) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_27 uid=23340 usec=116 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_28(unfulfilled) for JobId=8175006_28(8175034) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_28 uid=23340 usec=124 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_29(unfulfilled) for JobId=8175006_29(8175035) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_29 uid=23340 usec=116 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_30(unfulfilled) for JobId=8175006_30(8175036) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_30 uid=23340 usec=122 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_31(unfulfilled) for JobId=8175006_31(8175037) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_31 uid=23340 usec=122 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_32(unfulfilled) for JobId=8175006_32(8175038) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_32 uid=23340 usec=137 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_33(unfulfilled) for JobId=8175006_33(8175039) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_33 uid=23340 usec=113 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_34(unfulfilled) for JobId=8175006_34(8175040) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_34 uid=23340 usec=164 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_35(unfulfilled) for JobId=8175006_35(8175041) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_35 uid=23340 usec=133 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_36(unfulfilled) for JobId=8175006_36(8175042) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_36 uid=23340 usec=130 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_37(unfulfilled) for JobId=8175006_37(8175043) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_37 uid=23340 usec=168 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_38(unfulfilled) for JobId=8175006_38(8175044) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_38 uid=23340 usec=123 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_39(unfulfilled) for JobId=8175006_39(8175045) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_39 uid=23340 usec=126 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_40(unfulfilled) for JobId=8175006_40(8175046) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_40 uid=23340 usec=117 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_41(unfulfilled) for JobId=8175006_41(8175047) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_41 uid=23340 usec=195 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_42(unfulfilled) for JobId=8175006_42(8175048) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_42 uid=23340 usec=126 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_43(unfulfilled) for JobId=8175006_43(8175049) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_43 uid=23340 usec=144 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_44(unfulfilled) for JobId=8175006_44(8175050) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_44 uid=23340 usec=133 Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_45(unfulfilled) for JobId=8175006_45(8175051) Nov 05 11:46:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_45 uid=23340 usec=117 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_46(unfulfilled) for JobId=8175006_46(8175052) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_46 uid=23340 usec=20076 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_47(unfulfilled) for JobId=8175006_47(8175053) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_47 uid=23340 usec=320 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_48(unfulfilled) for JobId=8175006_48(8175054) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_48 uid=23340 usec=136 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_49(unfulfilled) for JobId=8175006_49(8175055) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_49 uid=23340 usec=117 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_50(unfulfilled) for JobId=8175006_50(8175056) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_50 uid=23340 usec=147 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_51(unfulfilled) for JobId=8175006_51(8175057) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_51 uid=23340 usec=118 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_52(unfulfilled) for JobId=8175006_52(8175058) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_52 uid=23340 usec=115 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_53(unfulfilled) for JobId=8175006_53(8175059) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_53 uid=23340 usec=113 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_54(unfulfilled) for JobId=8175006_54(8175060) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_54 uid=23340 usec=111 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_55(unfulfilled) for JobId=8175006_55(8175061) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_55 uid=23340 usec=118 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_56(unfulfilled) for JobId=8175006_56(8175062) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_56 uid=23340 usec=139 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_57(unfulfilled) for JobId=8175006_57(8175063) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_57 uid=23340 usec=117 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_58(unfulfilled) for JobId=8175006_58(8175064) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_58 uid=23340 usec=318 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_59(unfulfilled) for JobId=8175006_59(8175065) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_59 uid=23340 usec=122 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_60(unfulfilled) for JobId=8175006_60(8175066) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_60 uid=23340 usec=169 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_61(unfulfilled) for JobId=8175006_61(8175067) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_61 uid=23340 usec=35675 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_62(unfulfilled) for JobId=8175006_62(8175068) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_62 uid=23340 usec=150 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_63(unfulfilled) for JobId=8175006_63(8175069) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_63 uid=23340 usec=119 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_64(unfulfilled) for JobId=8175006_64(8175070) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_64 uid=23340 usec=113 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_65(unfulfilled) for JobId=8175006_65(8175071) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_65 uid=23340 usec=114 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_66(unfulfilled) for JobId=8175006_66(8175072) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_66 uid=23340 usec=112 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_67(unfulfilled) for JobId=8175006_67(8175073) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_67 uid=23340 usec=123 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_68(unfulfilled) for JobId=8175006_68(8175074) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_68 uid=23340 usec=116 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_69(unfulfilled) for JobId=8175006_69(8175075) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_69 uid=23340 usec=113 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_70(unfulfilled) for JobId=8175006_70(8175076) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_70 uid=23340 usec=113 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_71(unfulfilled) for JobId=8175006_71(8175077) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_71 uid=23340 usec=111 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_72(unfulfilled) for JobId=8175006_72(8175078) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_72 uid=23340 usec=114 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_73(unfulfilled) for JobId=8175006_73(8175079) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_73 uid=23340 usec=116 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_74(unfulfilled) for JobId=8175006_74(8175080) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_74 uid=23340 usec=123 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_75(unfulfilled) for JobId=8175006_75(8175081) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_75 uid=23340 usec=118 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_76(unfulfilled) for JobId=8175006_76(8175082) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_76 uid=23340 usec=115 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_77(unfulfilled) for JobId=8175006_77(8175083) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_77 uid=23340 usec=116 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_78(unfulfilled) for JobId=8175006_78(8175084) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_78 uid=23340 usec=113 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_79(unfulfilled) for JobId=8175006_79(8175085) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_79 uid=23340 usec=112 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_80(unfulfilled) for JobId=8175006_80(8175086) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_80 uid=23340 usec=116 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_81(unfulfilled) for JobId=8175006_81(8175087) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_81 uid=23340 usec=181 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_82(unfulfilled) for JobId=8175006_82(8175088) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_82 uid=23340 usec=150 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_83(unfulfilled) for JobId=8175006_83(8175089) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_83 uid=23340 usec=121 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_84(unfulfilled) for JobId=8175006_84(8175090) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_84 uid=23340 usec=118 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_85(unfulfilled) for JobId=8175006_85(8175091) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_85 uid=23340 usec=112 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_86(unfulfilled) for JobId=8175006_86(8175092) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_86 uid=23340 usec=112 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_87(unfulfilled) for JobId=8175006_87(8175093) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_87 uid=23340 usec=296 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_88(unfulfilled) for JobId=8175006_88(8175094) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_88 uid=23340 usec=110 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_89(unfulfilled) for JobId=8175006_89(8175095) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_89 uid=23340 usec=117 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_90(unfulfilled) for JobId=8175006_90(8175096) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_90 uid=23340 usec=120 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_91(unfulfilled) for JobId=8175006_91(8175097) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_91 uid=23340 usec=117 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_92(unfulfilled) for JobId=8175006_92(8175098) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_92 uid=23340 usec=115 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_93(unfulfilled) for JobId=8175006_93(8175099) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_93 uid=23340 usec=109 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_94(unfulfilled) for JobId=8175006_94(8175100) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_94 uid=23340 usec=163 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_95(unfulfilled) for JobId=8175006_95(8175101) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_95 uid=23340 usec=110 Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8174910_96(unfulfilled) for JobId=8175006_96(8175006) Nov 05 11:46:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175006_96 uid=23340 usec=165 Nov 05 11:46:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7678(8168314) WEXITSTATUS 1 Nov 05 11:46:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7678(8168314) done Nov 05 11:46:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133397 WEXITSTATUS 0 Nov 05 11:46:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133397 done Nov 05 11:46:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170256_2(8170286) WEXITSTATUS 0 Nov 05 11:46:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170256_2(8170286) done Nov 05 11:46:46 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175102 NodeList=(null) usec=21644 Nov 05 11:46:47 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175102 NodeList=r806u25n01 #CPUs=1 Partition=devel Nov 05 11:46:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_982(8115251) Nov 05 11:46:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8115470_88(8115558) Nov 05 11:46:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8115470_505(8115978) Nov 05 11:46:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8115470_575(8116048) Nov 05 11:46:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 11:46:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170256_3(8175103) NodeList=r4512u32n01 #CPUs=1 Partition=transfer Nov 05 11:47:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175104 NodeList=(null) usec=34701 Nov 05 11:47:01 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175104 NodeList=r806u25n01 #CPUs=1 Partition=devel Nov 05 11:47:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135450 WEXITSTATUS 0 Nov 05 11:47:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135450 done Nov 05 11:47:01 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u32n01 Nov 05 11:47:01 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u32n03 Nov 05 11:47:01 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u32n02 Nov 05 11:47:01 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u32n04 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:47:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7931(8170455) WEXITSTATUS 1 Nov 05 11:47:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7931(8170455) done Nov 05 11:47:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8126(8172833) WEXITSTATUS 1 Nov 05 11:47:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8126(8172833) done Nov 05 11:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133398 WEXITSTATUS 0 Nov 05 11:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133398 done Nov 05 11:47:24 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175105 NodeList=(null) usec=32806 Nov 05 11:47:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175105 NodeList=r806u25n01 #CPUs=4 Partition=devel Nov 05 11:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166343 WEXITSTATUS 0 Nov 05 11:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166343 done Nov 05 11:47:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174107 in gpu on r915u13n01 Nov 05 11:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8054(8172492) WEXITSTATUS 1 Nov 05 11:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8054(8172492) done Nov 05 11:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1165(7570827) WEXITSTATUS 0 Nov 05 11:47:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7569662_1165(7570827) done Nov 05 11:47:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166589 in gpu on r908u24n01 Nov 05 11:47:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166590 in gpu on r908u24n02 Nov 05 11:47:54 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 11:47:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8181(8175106) in scavenge on r802u06n04 Nov 05 11:47:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8182(8175107) in scavenge on r816u29n09 Nov 05 11:47:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8183(8175108) in scavenge on r816u09n09 Nov 05 11:47:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8184(8175109) in scavenge on r816u15n11 Nov 05 11:47:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8185(8175110) in scavenge on r805u09n04 Nov 05 11:47:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8186(8175111) in scavenge on r807u05n02 Nov 05 11:47:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8187(8175112) in scavenge on r918u09n03 Nov 05 11:47:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_201(8114467) in scavenge on r807u37n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_1(8175113) in scavenge on r908u28n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_2(8175114) in scavenge on r907u01n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_3(8175115) in scavenge on r907u01n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_4(8175116) in scavenge on r907u01n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_5(8175117) in scavenge on r907u01n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_6(8175118) in scavenge on r907u01n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_7(8175119) in scavenge on r907u01n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_8(8175120) in scavenge on r907u05n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_9(8175121) in scavenge on r907u05n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_10(8175122) in scavenge on r907u05n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_11(8175123) in scavenge on r907u05n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_12(8175124) in scavenge on r907u05n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_13(8175125) in scavenge on r907u05n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_14(8175126) in scavenge on r907u05n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_15(8175127) in scavenge on r907u05n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_16(8175128) in scavenge on r801u11n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_17(8175129) in scavenge on r802u30n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_18(8175130) in scavenge on r802u30n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_19(8175131) in scavenge on r915u35n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_20(8175132) in scavenge on r917u10n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_21(8175133) in scavenge on r911u09n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_22(8175134) in scavenge on r911u09n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_23(8175135) in scavenge on r911u28n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_24(8175136) in scavenge on r911u28n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_25(8175137) in scavenge on r911u28n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_26(8175138) in scavenge on r911u28n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_27(8175139) in scavenge on r911u28n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_28(8175140) in scavenge on r815u03n10 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_29(8175141) in scavenge on r815u03n10 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_30(8175142) in scavenge on r816u23n08 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_31(8175143) in scavenge on r903u34n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_32(8175144) in scavenge on r808u14n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_33(8175145) in scavenge on r808u17n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_34(8175146) in scavenge on r4513u23n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_35(8175147) in scavenge on r902u07n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_36(8175148) in scavenge on r902u07n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_37(8175149) in scavenge on r907u32n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_38(8175150) in scavenge on r906u28n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_39(8175151) in scavenge on r807u03n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_40(8175152) in scavenge on r912u28n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_41(8175153) in scavenge on r805u11n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_42(8175154) in scavenge on r805u23n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_43(8175155) in scavenge on r909u07n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_44(8175156) in scavenge on r909u10n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_45(8175157) in scavenge on r918u05n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_46(8175158) in scavenge on r918u09n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_47(8175159) in scavenge on r918u09n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_48(8175160) in scavenge on r918u09n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_49(8175161) in scavenge on r919u01n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_50(8175162) in scavenge on r919u01n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_51(8175163) in scavenge on r806u06n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_52(8175164) in scavenge on r806u09n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_53(8175165) in scavenge on r806u11n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_54(8175166) in scavenge on r806u11n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_55(8175167) in scavenge on r806u11n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_56(8175168) in scavenge on r806u11n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_57(8175169) in scavenge on r816u15n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_58(8175170) in scavenge on r816u29n05 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_59(8175171) in scavenge on r815u29n03 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_60(8175172) in scavenge on r815u35n04 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_61(8175173) in scavenge on r815u03n08 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_62(8175174) in scavenge on r815u09n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_63(8175175) in scavenge on r815u09n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_64(8175176) in scavenge on r815u15n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_65(8175177) in scavenge on r807u37n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_66(8175178) in scavenge on r908u24n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_67(8175179) in scavenge on r908u24n02 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_68(8175180) in scavenge on r915u09n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_69(8175181) in scavenge on r806u35n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_70(8175182) in scavenge on r806u35n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_71(8175183) in scavenge on r806u35n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_72(8175184) in scavenge on r806u37n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_73(8175185) in scavenge on r805u32n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_74(8175186) in scavenge on r805u32n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_75(8175187) in scavenge on r805u34n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_76(8175188) in scavenge on r805u34n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_77(8175189) in scavenge on r805u34n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_78(8175190) in scavenge on r805u36n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174141_79(8175191) in scavenge on r805u36n01 Nov 05 11:48:01 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:158 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 11:48:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2965(8175192) in scavenge on r807u37n01 Nov 05 11:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170113 WEXITSTATUS 0 Nov 05 11:48:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170113 done Nov 05 11:48:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175193 NodeList=(null) usec=28349 Nov 05 11:48:09 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175193 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 11:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2342(8152259) WEXITSTATUS 0 Nov 05 11:48:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8146707_2342(8152259) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_33(8175145) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_33(8175145) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_57(8175169) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_57(8175169) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_36(8175148) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_36(8175148) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_35(8175147) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_35(8175147) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_34(8175146) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_34(8175146) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_65(8175177) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_65(8175177) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_31(8175143) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_31(8175143) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_61(8175173) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_61(8175173) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_30(8175142) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_30(8175142) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_32(8175144) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_32(8175144) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_62(8175174) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_62(8175174) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_63(8175175) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_63(8175175) done Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_64(8175176) WEXITSTATUS 1 Nov 05 11:48:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_64(8175176) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_28(8175140) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_28(8175140) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_29(8175141) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_29(8175141) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_59(8175171) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_59(8175171) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_60(8175172) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_60(8175172) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_68(8175180) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_68(8175180) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_58(8175170) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_58(8175170) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_9(8175121) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_9(8175121) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_44(8175156) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_44(8175156) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_10(8175122) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_10(8175122) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_11(8175123) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_11(8175123) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_45(8175157) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_45(8175157) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_7(8175119) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_7(8175119) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_4(8175116) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_4(8175116) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_6(8175118) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_6(8175118) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_43(8175155) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_43(8175155) done Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_5(8175117) WEXITSTATUS 1 Nov 05 11:48:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_5(8175117) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_13(8175125) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_13(8175125) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_12(8175124) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_12(8175124) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_14(8175126) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_14(8175126) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_15(8175127) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_15(8175127) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_8(8175120) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_8(8175120) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_49(8175161) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_49(8175161) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_50(8175162) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_50(8175162) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_2(8175114) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_2(8175114) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_3(8175115) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_3(8175115) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_47(8175159) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_47(8175159) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_48(8175160) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_48(8175160) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_46(8175158) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_46(8175158) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_66(8175178) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_66(8175178) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_1(8175113) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_1(8175113) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_67(8175179) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_67(8175179) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_19(8175131) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_19(8175131) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_22(8175134) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_22(8175134) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_40(8175152) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_40(8175152) done Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_20(8175132) WEXITSTATUS 1 Nov 05 11:48:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_20(8175132) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_37(8175149) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_37(8175149) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_41(8175153) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_41(8175153) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_51(8175163) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_51(8175163) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_42(8175154) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_42(8175154) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_16(8175128) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_16(8175128) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_38(8175150) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_38(8175150) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_21(8175133) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_21(8175133) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_24(8175136) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_24(8175136) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_23(8175135) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_23(8175135) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_17(8175129) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_17(8175129) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_79(8175191) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_79(8175191) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_78(8175190) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_78(8175190) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_52(8175164) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_52(8175164) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_56(8175168) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_56(8175168) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_55(8175167) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_55(8175167) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_70(8175182) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_70(8175182) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_69(8175181) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_69(8175181) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_53(8175165) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_53(8175165) done Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_71(8175183) WEXITSTATUS 1 Nov 05 11:48:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_71(8175183) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_39(8175151) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_39(8175151) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_54(8175166) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_54(8175166) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_73(8175185) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_73(8175185) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_74(8175186) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_74(8175186) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_72(8175184) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_72(8175184) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_77(8175189) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_77(8175189) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_18(8175130) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_18(8175130) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5955(8162951) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5955(8162951) done Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_75(8175187) WEXITSTATUS 1 Nov 05 11:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_75(8175187) done Nov 05 11:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_25(8175137) WEXITSTATUS 1 Nov 05 11:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_25(8175137) done Nov 05 11:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_26(8175138) WEXITSTATUS 1 Nov 05 11:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_26(8175138) done Nov 05 11:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_27(8175139) WEXITSTATUS 1 Nov 05 11:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_27(8175139) done Nov 05 11:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_76(8175188) WEXITSTATUS 1 Nov 05 11:48:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174141_76(8175188) done Nov 05 11:48:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7507(8166499) WEXITSTATUS 1 Nov 05 11:48:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7507(8166499) done Nov 05 11:48:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7572(8166810) WEXITSTATUS 1 Nov 05 11:48:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7572(8166810) done Nov 05 11:48:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175194 InitPrio=62986 usec=3976 Nov 05 11:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4011(8172666) WEXITSTATUS 0 Nov 05 11:48:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4011(8172666) done Nov 05 11:48:37 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u13n01.grace.ycrc.yale.edu" Nov 05 11:48:37 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u13n01.grace.ycrc.yale.edu" Nov 05 11:48:37 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u13n02.grace.ycrc.yale.edu" Nov 05 11:48:37 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u13n02.grace.ycrc.yale.edu" Nov 05 11:48:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:48:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175195 InitPrio=24888 usec=14098 Nov 05 11:48:38 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u09n04.grace.ycrc.yale.edu" Nov 05 11:48:38 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u09n04.grace.ycrc.yale.edu" Nov 05 11:48:40 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u09n03.grace.ycrc.yale.edu" Nov 05 11:48:40 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u09n03.grace.ycrc.yale.edu" Nov 05 11:48:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175196 InitPrio=26160 usec=3817 Nov 05 11:48:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175196 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 11:48:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:48:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172432 WEXITSTATUS 0 Nov 05 11:48:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172432 done Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174142_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174238_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174334_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174430_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174526_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174622_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174718_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174814_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_93 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174910_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_5 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_2 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_6 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_4 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_1 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_14 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_10 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_11 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_17 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_9 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_3 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_20 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_8 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_12 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_18 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_24 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_25 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_22 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_7 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_19 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_15 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_21 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_26 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_23 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_16 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_29 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_35 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_31 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_32 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_33 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_34 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_36 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_28 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_13 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_43 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_40 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_45 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_42 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_30 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_39 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_49 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_37 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_41 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_38 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_27 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_44 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_55 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_52 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_53 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_54 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_46 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_51 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_47 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_57 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_58 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_60 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_65 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_48 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_67 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_63 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_64 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_70 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_71 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_50 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_68 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_69 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_62 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_76 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_77 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_73 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_61 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_74 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_75 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_59 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_78 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_80 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_81 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_56 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_83 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_84 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_72 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_82 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_79 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_66 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_85 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_90 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_91 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_88 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_89 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_94 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_95 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_92 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8174141_[80-96] uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_86 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_96 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_87 uid 23340 Nov 05 11:48:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175006_93 uid 23340 Nov 05 11:48:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4008(8172571) WEXITSTATUS 0 Nov 05 11:48:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4008(8172571) done Nov 05 11:48:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:48:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:48:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175194 NodeList=r806u28n01 #CPUs=1 Partition=bigmem Nov 05 11:48:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6293(8163646) WEXITSTATUS 1 Nov 05 11:48:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6293(8163646) done Nov 05 11:48:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:48:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7292(8165968) WEXITSTATUS 1 Nov 05 11:48:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7292(8165968) done Nov 05 11:49:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7909(8170208) WEXITSTATUS 1 Nov 05 11:49:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7909(8170208) done Nov 05 11:49:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164702 WEXITSTATUS 0 Nov 05 11:49:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8164702 done Nov 05 11:49:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175197 InitPrio=17501 usec=5562 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:49:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7841938 WEXITSTATUS 127 Nov 05 11:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7841938 done Nov 05 11:49:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:21 r912u23n01 slurmctld[163760]: slurmctld: _pick_best_nodes: JobId=8175198 never runnable in partition pi_panda Nov 05 11:49:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_allocate_resources: Requested node configuration is not available Nov 05 11:49:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_614(7572783) Nov 05 11:49:22 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_342(8114609) Nov 05 11:49:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:28 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175199 NodeList=(null) usec=32801 Nov 05 11:49:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_812(7572981) WEXITSTATUS 0 Nov 05 11:49:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_812(7572981) done Nov 05 11:49:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7725(8168716) WEXITSTATUS 1 Nov 05 11:49:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7725(8168716) done Nov 05 11:49:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175200 InitPrio=36107 usec=4185 Nov 05 11:49:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175200 NodeList=r806u25n02 #CPUs=1 Partition=devel Nov 05 11:49:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175200 uid 25892 Nov 05 11:49:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:50 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25892: QOSMaxMemoryPerUser Nov 05 11:49:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:49:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172917 WEXITSTATUS 0 Nov 05 11:49:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172917 done Nov 05 11:49:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3302(8134661) Nov 05 11:49:52 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3303(8134662) Nov 05 11:49:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:53 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175197_0(8175197) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 11:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25892: QOSMaxMemoryPerUser Nov 05 11:49:54 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:49:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 25892: QOSMaxMemoryPerUser Nov 05 11:49:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:49:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175195 uid 19353 Nov 05 11:49:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7619(8167064) WEXITSTATUS 1 Nov 05 11:49:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7619(8167064) done Nov 05 11:50:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175201 InitPrio=36107 usec=3622 Nov 05 11:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175201 NodeList=r806u25n02 #CPUs=1 Partition=devel Nov 05 11:50:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175199 WTERMSIG 2 Nov 05 11:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175199 cancelled by interactive user Nov 05 11:50:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175199 done Nov 05 11:50:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8175199 error Job/step already completing or completed Nov 05 11:50:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175202 InitPrio=24888 usec=3758 Nov 05 11:50:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175202 NodeList=r903u01n02 #CPUs=1 Partition=pi_zhu Nov 05 11:50:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175202 WEXITSTATUS 1 Nov 05 11:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175202 done Nov 05 11:50:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175203 NodeList=(null) usec=29549 Nov 05 11:50:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175203 NodeList=r906u39n01 #CPUs=8 Partition=pi_panda Nov 05 11:50:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174107 WEXITSTATUS 1 Nov 05 11:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174107 done Nov 05 11:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8147(8172910) WEXITSTATUS 1 Nov 05 11:50:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8147(8172910) done Nov 05 11:50:18 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u32n04.grace.ycrc.yale.edu" Nov 05 11:50:18 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u32n04.grace.ycrc.yale.edu" Nov 05 11:50:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:20 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u32n01.grace.ycrc.yale.edu" Nov 05 11:50:20 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u32n01.grace.ycrc.yale.edu" Nov 05 11:50:21 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6571(8164608) has been requeued to reclaim resources for JobId=Invalid Nov 05 11:50:21 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u32n02.grace.ycrc.yale.edu" Nov 05 11:50:21 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u32n02.grace.ycrc.yale.edu" Nov 05 11:50:22 r912u23n01 slurmctld[163760]: slurmctld: node r905u13n01 returned to service Nov 05 11:50:22 r912u23n01 slurmctld[163760]: slurmctld: node r905u13n02 returned to service Nov 05 11:50:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6571(8164608) Nov 05 11:50:22 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u32n03.grace.ycrc.yale.edu" Nov 05 11:50:22 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u32n03.grace.ycrc.yale.edu" Nov 05 11:50:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:23 r912u23n01 slurmctld[163760]: slurmctld: node r905u09n04 returned to service Nov 05 11:50:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:25 r912u23n01 slurmctld[163760]: slurmctld: node r905u09n03 returned to service Nov 05 11:50:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175204 InitPrio=24888 usec=3701 Nov 05 11:50:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175204 NodeList=r903u01n02 #CPUs=1 Partition=pi_zhu Nov 05 11:50:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:50:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_17(8175205) in day on r816u09n08 Nov 05 11:50:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4022(8175206) in gpu on r808u17n01 Nov 05 11:50:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4023(8175207) in gpu on r903u34n01 Nov 05 11:50:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u03n02 Nov 05 11:50:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_876(7573045) WEXITSTATUS 0 Nov 05 11:50:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_876(7573045) done Nov 05 11:50:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175204 WEXITSTATUS 0 Nov 05 11:50:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175204 done Nov 05 11:50:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154671 WEXITSTATUS 0 Nov 05 11:50:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154671 done Nov 05 11:50:49 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174105 NodeList=r906u41n02 #CPUs=6 Partition=day Nov 05 11:50:51 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8164465 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8188(8175208) in scavenge on r907u05n04 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8189(8175209) in scavenge on r911u01n01 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_8190(8112304) in scavenge on r916u04n02 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2696(8128067) in scavenge on r915u38n03 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_894(8122839) in scavenge on r802u02n02 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_455(8113633) in scavenge on r802u32n03 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_570(8113950) in scavenge on r816u03n09 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_392(8113275) in scavenge on r816u23n07 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1081(8123109) in scavenge on r906u39n01 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1458(8123698) in scavenge on r805u27n01 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1223(8123365) in scavenge on r805u27n01 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1492(8123744) in scavenge on r909u01n03 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_476(8114743) in scavenge on r902u10n01 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_378(8115848) in scavenge on r806u35n01 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8133(8172840) WEXITSTATUS 1 Nov 05 11:50:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8133(8172840) done Nov 05 11:51:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2966(8175210) in scavenge on r807u37n01 Nov 05 11:51:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2967(8175211) in scavenge on r807u37n01 Nov 05 11:51:01 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2968(8175212) in scavenge on r915u13n01 Nov 05 11:51:02 r912u23n01 slurmctld[163760]: slurmctld: error: slurm_send_node_msg: [socket:[2269687564]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 05 11:51:02 r912u23n01 slurmctld[163760]: error: slurm_send_node_msg: [socket:[2269687564]] slurm_bufs_sendto(msg_type=SRUN_STEP_SIGNAL) failed: Connection reset by peer Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:51:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:51:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175213 InitPrio=16342 usec=7677 Nov 05 11:51:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8092(8172679) WEXITSTATUS 1 Nov 05 11:51:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8092(8172679) done Nov 05 11:51:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175214 InitPrio=16342 usec=4102 Nov 05 11:51:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4831(8157810) WEXITSTATUS 1 Nov 05 11:51:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4831(8157810) done Nov 05 11:51:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7934(8170517) WEXITSTATUS 1 Nov 05 11:51:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7934(8170517) done Nov 05 11:51:50 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175215 InitPrio=25725 usec=3743 Nov 05 11:51:53 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:52:03 r912u23n01 slurmctld[163760]: slurmctld: node r905u32n04 returned to service Nov 05 11:52:04 r912u23n01 slurmctld[163760]: slurmctld: node r905u32n01 returned to service Nov 05 11:52:06 r912u23n01 slurmctld[163760]: slurmctld: node r905u32n02 returned to service Nov 05 11:52:07 r912u23n01 slurmctld[163760]: slurmctld: node r905u32n03 returned to service Nov 05 11:52:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175216 NodeList=(null) usec=32098 Nov 05 11:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175216 WTERMSIG 2 Nov 05 11:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175216 cancelled by interactive user Nov 05 11:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175216 done Nov 05 11:52:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8175216 error Job/step already completing or completed Nov 05 11:52:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175217 NodeList=(null) usec=23402 Nov 05 11:52:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175217 NodeList=r909u29n01 #CPUs=1 Partition=day Nov 05 11:52:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8176(8174115) WEXITSTATUS 1 Nov 05 11:52:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8176(8174115) done Nov 05 11:52:21 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7569662_726(7570388) Nov 05 11:52:21 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_499(7572667) Nov 05 11:52:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175218 InitPrio=16342 usec=3992 Nov 05 11:52:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4017(8172832) WEXITSTATUS 0 Nov 05 11:52:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4017(8172832) done Nov 05 11:52:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8023(8171990) WEXITSTATUS 1 Nov 05 11:52:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8023(8171990) done Nov 05 11:52:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:52:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175219 InitPrio=25500 usec=7620 Nov 05 11:52:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3777(8164667) WEXITSTATUS 0 Nov 05 11:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3777(8164667) done Nov 05 11:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7611(8167056) WEXITSTATUS 1 Nov 05 11:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7611(8167056) done Nov 05 11:52:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:52:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:52:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8153362 NodeList=r803u04n[03-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-02],r905u05n[01-04],r905u09n[01-04],r905u13n[01-02],r905u32n[01-04],r906u05n[03-04],r906u09n[01-04] #CPUs=1536 Partition=scavenge_mpi Nov 05 11:52:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175197_0(8175197) WEXITSTATUS 0 Nov 05 11:52:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175197_0(8175197) done Nov 05 11:53:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:53:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:53:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2443(8165717) WEXITSTATUS 0 Nov 05 11:53:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2443(8165717) done Nov 05 11:53:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175220 InitPrio=24842 usec=3925 Nov 05 11:53:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175221 InitPrio=24840 usec=4073 Nov 05 11:53:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175222 InitPrio=24842 usec=3099 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175223 InitPrio=24840 usec=2990 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175224 InitPrio=24842 usec=3224 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175225 InitPrio=24840 usec=2552 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175226 InitPrio=24842 usec=2384 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175227 InitPrio=24840 usec=4059 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175228 InitPrio=24842 usec=3390 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175229 InitPrio=24840 usec=3470 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175230 InitPrio=24842 usec=2679 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175231 InitPrio=24840 usec=2404 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175232 InitPrio=24842 usec=2635 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175233 InitPrio=24840 usec=2592 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175234 InitPrio=24842 usec=2410 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175235 InitPrio=24840 usec=2355 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175236 InitPrio=24842 usec=2406 Nov 05 11:53:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175237 InitPrio=24840 usec=2331 Nov 05 11:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7495(8166468) WEXITSTATUS 1 Nov 05 11:53:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7495(8166468) done Nov 05 11:53:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175203 WEXITSTATUS 130 Nov 05 11:53:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175203 done Nov 05 11:53:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175238 InitPrio=20736 usec=5701 Nov 05 11:53:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175215 in day on r805u04n02 Nov 05 11:53:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175220 in mpi on r803u02n[01-04],r803u04n01 Nov 05 11:53:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_18(8175239) in day on r806u11n03 Nov 05 11:53:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4024(8175240) in gpu on r808u14n01 Nov 05 11:53:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4025(8175241) in gpu on r4513u23n01 Nov 05 11:53:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_913(7573082) WEXITSTATUS 0 Nov 05 11:53:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_913(7573082) done Nov 05 11:53:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_215(8114481) WEXITSTATUS 0 Nov 05 11:53:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_215(8114481) done Nov 05 11:53:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_725(7572894) WEXITSTATUS 0 Nov 05 11:53:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_725(7572894) done Nov 05 11:53:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175242 InitPrio=16342 usec=4338 Nov 05 11:53:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175105 WEXITSTATUS 0 Nov 05 11:53:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175105 done Nov 05 11:53:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 11:53:51 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175243 NodeList=(null) usec=741914 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_721(8114202) in scavenge on r906u01n02 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1597(8124141) in scavenge on r912u01n02 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3612(8136977) in scavenge on r808u17n01 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3686(8138482) in scavenge on r805u23n01 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1623(8124198) in scavenge on r806u09n03 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2165(8125998) in scavenge on r909u04n02 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1900(8125196) in scavenge on r909u04n04 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1913(8125242) in scavenge on r918u09n03 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_444(7571409) in scavenge on r816u15n09 Nov 05 11:53:52 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_711(7572880) in scavenge on r816u23n05 Nov 05 11:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_476(8114743) WEXITSTATUS 0 Nov 05 11:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_476(8114743) done Nov 05 11:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7784(8169192) WEXITSTATUS 1 Nov 05 11:53:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7784(8169192) done Nov 05 11:53:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175243 NodeList=r902u10n01 #CPUs=8 Partition=pi_panda Nov 05 11:53:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2969(8175244) in scavenge on r808u14n01 Nov 05 11:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7053(8165500) WEXITSTATUS 1 Nov 05 11:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7053(8165500) done Nov 05 11:54:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175245 InitPrio=59397 usec=5348 Nov 05 11:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7154(8165759) WEXITSTATUS 1 Nov 05 11:54:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7154(8165759) done Nov 05 11:54:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133399 WEXITSTATUS 0 Nov 05 11:54:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133399 done Nov 05 11:54:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8015(8171816) WEXITSTATUS 1 Nov 05 11:54:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8015(8171816) done Nov 05 11:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8017(8171818) WEXITSTATUS 1 Nov 05 11:54:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8017(8171818) done Nov 05 11:54:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4025(8175241) WEXITSTATUS 0 Nov 05 11:54:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4025(8175241) done Nov 05 11:54:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133400 WEXITSTATUS 0 Nov 05 11:54:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8133400 done Nov 05 11:54:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8170(8174010) WEXITSTATUS 1 Nov 05 11:54:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8170(8174010) done Nov 05 11:54:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7844(8169561) WEXITSTATUS 1 Nov 05 11:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7844(8169561) done Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175246 InitPrio=17501 usec=3299 Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175247 InitPrio=17501 usec=2812 Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175248 InitPrio=17501 usec=2400 Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_28(8165384) WEXITSTATUS 0 Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_28(8165384) done Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175249 InitPrio=17501 usec=3252 Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175250 InitPrio=17501 usec=2881 Nov 05 11:54:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175251 InitPrio=17501 usec=2788 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175252 InitPrio=17501 usec=2661 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175253 InitPrio=17501 usec=3351 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175254 InitPrio=17501 usec=3067 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175255 InitPrio=17501 usec=3009 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175256 InitPrio=17501 usec=2721 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175257 InitPrio=17501 usec=2508 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175258 InitPrio=17501 usec=3395 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175259 InitPrio=17501 usec=3065 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175260 InitPrio=17501 usec=2658 Nov 05 11:54:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175261 InitPrio=17501 usec=2913 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175262 InitPrio=17501 usec=3382 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175263 InitPrio=17501 usec=2798 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175264 InitPrio=17501 usec=2614 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175265 InitPrio=17501 usec=3503 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175266 InitPrio=17501 usec=3317 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175267 InitPrio=17501 usec=3106 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175268 InitPrio=17501 usec=4183 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175269 InitPrio=17501 usec=3837 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175270 InitPrio=17501 usec=5805 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175271 InitPrio=17501 usec=4729 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175272 InitPrio=17501 usec=4839 Nov 05 11:54:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175273 InitPrio=17501 usec=3894 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175274 InitPrio=17501 usec=4307 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175275 InitPrio=17501 usec=3710 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175276 InitPrio=17501 usec=2855 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175277 InitPrio=17501 usec=3929 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175278 InitPrio=17501 usec=2932 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175279 InitPrio=17501 usec=3924 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175280 InitPrio=17501 usec=3643 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175281 InitPrio=17501 usec=3278 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175282 InitPrio=17501 usec=3398 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175283 InitPrio=17501 usec=3028 Nov 05 11:54:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175284 InitPrio=17501 usec=3427 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175285 InitPrio=17501 usec=3496 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175286 InitPrio=17501 usec=3224 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175287 InitPrio=17501 usec=2706 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175288 InitPrio=17501 usec=2379 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175289 InitPrio=17501 usec=3090 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175290 InitPrio=17501 usec=2637 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175291 InitPrio=17501 usec=3197 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175292 InitPrio=17501 usec=3566 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175293 InitPrio=17501 usec=2902 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175294 InitPrio=17501 usec=2366 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175295 InitPrio=17501 usec=2247 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175296 InitPrio=17501 usec=2659 Nov 05 11:54:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175297 InitPrio=17501 usec=3160 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175298 InitPrio=17501 usec=2607 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175299 InitPrio=17501 usec=2940 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175300 InitPrio=17501 usec=2575 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175301 InitPrio=17501 usec=3486 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175302 InitPrio=17501 usec=2989 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175303 InitPrio=17501 usec=2532 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175304 InitPrio=17501 usec=2910 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175305 InitPrio=17501 usec=2643 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175306 InitPrio=17501 usec=2353 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175307 InitPrio=17501 usec=2181 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175308 InitPrio=17501 usec=3427 Nov 05 11:54:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175309 InitPrio=17501 usec=2891 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175310 InitPrio=17501 usec=3065 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175311 InitPrio=17501 usec=3585 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175312 InitPrio=17501 usec=2814 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175313 InitPrio=17501 usec=1899 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175314 InitPrio=17501 usec=1859 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175315 InitPrio=17501 usec=2044 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175316 InitPrio=17501 usec=4395 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175317 InitPrio=17501 usec=3477 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175318 InitPrio=17501 usec=2295 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175319 InitPrio=17501 usec=2084 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175320 InitPrio=17501 usec=3098 Nov 05 11:54:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175321 InitPrio=17501 usec=2520 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175322 InitPrio=17501 usec=1953 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175323 InitPrio=17501 usec=3873 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175324 InitPrio=17501 usec=3250 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175325 InitPrio=17501 usec=2082 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175326 InitPrio=17501 usec=3114 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175327 InitPrio=17501 usec=2388 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175328 InitPrio=17501 usec=2172 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175329 InitPrio=17501 usec=2090 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175330 InitPrio=17501 usec=3257 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175331 InitPrio=17501 usec=2479 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175332 InitPrio=17501 usec=2226 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175333 InitPrio=17501 usec=3635 Nov 05 11:54:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175334 InitPrio=17501 usec=2861 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175335 InitPrio=17501 usec=2573 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175336 InitPrio=17501 usec=3703 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175337 InitPrio=17501 usec=3406 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175338 InitPrio=17501 usec=5329 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175339 InitPrio=17501 usec=3521 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175340 InitPrio=17501 usec=3117 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175341 InitPrio=17501 usec=3000 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175342 InitPrio=17501 usec=3727 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175343 InitPrio=17501 usec=3551 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175344 InitPrio=17501 usec=2625 Nov 05 11:54:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175345 InitPrio=17501 usec=2209 Nov 05 11:54:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175220 WEXITSTATUS 0 Nov 05 11:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175220 done Nov 05 11:54:51 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u02n02 Nov 05 11:54:51 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u02n03 Nov 05 11:54:51 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u04n01 Nov 05 11:54:51 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u02n04 Nov 05 11:54:51 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r803u02n01 Nov 05 11:54:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175245 uid 25356 Nov 05 11:54:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:54:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175246_0(8175246) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 11:54:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7326(8166017) WEXITSTATUS 1 Nov 05 11:54:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7326(8166017) done Nov 05 11:54:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:55:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:55:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175346 InitPrio=17501 usec=5110 Nov 05 11:55:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175347 InitPrio=17501 usec=4698 Nov 05 11:55:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175348 InitPrio=17501 usec=3700 Nov 05 11:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144134 WEXITSTATUS 0 Nov 05 11:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144134 done Nov 05 11:55:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175349 InitPrio=17501 usec=4331 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175350 InitPrio=17501 usec=2878 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175351 InitPrio=17501 usec=2480 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175352 InitPrio=17501 usec=2252 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175353 InitPrio=17501 usec=2437 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175354 InitPrio=17501 usec=3605 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175355 InitPrio=17501 usec=2273 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175356 InitPrio=17501 usec=2129 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175357 InitPrio=17501 usec=3352 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175358 InitPrio=17501 usec=2835 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175359 InitPrio=17501 usec=3670 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175360 InitPrio=17501 usec=2347 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175361 InitPrio=17501 usec=3152 Nov 05 11:55:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175362 InitPrio=17501 usec=2066 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175363 InitPrio=17501 usec=3439 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175364 InitPrio=17501 usec=3797 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175365 InitPrio=17501 usec=3501 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175366 InitPrio=17501 usec=3339 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175367 InitPrio=17501 usec=2881 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175368 InitPrio=17501 usec=3053 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175369 InitPrio=17501 usec=2470 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175370 InitPrio=17501 usec=2235 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175371 InitPrio=17501 usec=2161 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7642(8167110) WEXITSTATUS 1 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7642(8167110) done Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175372 InitPrio=17501 usec=2274 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175373 InitPrio=17501 usec=2063 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175374 InitPrio=17501 usec=1910 Nov 05 11:55:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175375 InitPrio=17501 usec=1898 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175376 InitPrio=17501 usec=3607 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175377 InitPrio=17501 usec=4497 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175378 InitPrio=17501 usec=3503 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175379 InitPrio=17501 usec=2153 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175380 InitPrio=17501 usec=2023 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175381 InitPrio=17501 usec=3618 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175382 InitPrio=17501 usec=2970 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175383 InitPrio=17501 usec=2132 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175384 InitPrio=17501 usec=2571 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175385 InitPrio=17501 usec=2159 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175386 InitPrio=17501 usec=2404 Nov 05 11:55:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175387 InitPrio=17501 usec=2653 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175388 InitPrio=17501 usec=2245 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175389 InitPrio=17501 usec=2982 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175390 InitPrio=17501 usec=3168 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175391 InitPrio=17501 usec=3345 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175392 InitPrio=17501 usec=2567 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175393 InitPrio=17501 usec=2324 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175394 InitPrio=17501 usec=2440 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175395 InitPrio=17501 usec=2069 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175396 InitPrio=17501 usec=2071 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175397 InitPrio=17501 usec=3312 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175398 InitPrio=17501 usec=3586 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175399 InitPrio=17501 usec=2880 Nov 05 11:55:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175400 InitPrio=17501 usec=2051 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175401 InitPrio=17501 usec=3595 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175402 InitPrio=17501 usec=4419 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175403 InitPrio=17501 usec=3514 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175404 InitPrio=17501 usec=3023 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175405 InitPrio=17501 usec=2931 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175406 InitPrio=17501 usec=3545 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175407 InitPrio=17501 usec=2745 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175408 InitPrio=17501 usec=2222 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175409 InitPrio=17501 usec=2900 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175410 InitPrio=17501 usec=2702 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175411 InitPrio=17501 usec=2988 Nov 05 11:55:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175412 InitPrio=17501 usec=2376 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175413 InitPrio=17501 usec=2141 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175414 InitPrio=17501 usec=2887 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175415 InitPrio=17501 usec=2207 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175416 InitPrio=17501 usec=1897 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175417 InitPrio=17501 usec=1804 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175418 InitPrio=17501 usec=1972 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175419 InitPrio=17501 usec=3676 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175420 InitPrio=17501 usec=2012 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175421 InitPrio=17501 usec=2129 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175422 InitPrio=17501 usec=3157 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175423 InitPrio=17501 usec=2157 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175424 InitPrio=17501 usec=2239 Nov 05 11:55:16 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175425 InitPrio=17501 usec=3436 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175426 InitPrio=17501 usec=2756 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175427 InitPrio=17501 usec=2566 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175428 InitPrio=17501 usec=2010 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175429 InitPrio=17501 usec=3988 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175430 InitPrio=17501 usec=1994 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175431 InitPrio=17501 usec=1887 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175432 InitPrio=17501 usec=3414 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175433 InitPrio=17501 usec=2381 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175434 InitPrio=17501 usec=2821 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175435 InitPrio=17501 usec=2304 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175436 InitPrio=17501 usec=3693 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175437 InitPrio=17501 usec=1876 Nov 05 11:55:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175438 InitPrio=17501 usec=1896 Nov 05 11:55:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175439 InitPrio=17501 usec=1919 Nov 05 11:55:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175440 InitPrio=17501 usec=3971 Nov 05 11:55:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175441 InitPrio=17501 usec=2996 Nov 05 11:55:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175442 InitPrio=17501 usec=2092 Nov 05 11:55:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175443 InitPrio=17501 usec=3478 Nov 05 11:55:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175444 InitPrio=17501 usec=2390 Nov 05 11:55:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175445 InitPrio=17501 usec=2083 Nov 05 11:55:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175446 InitPrio=59397 usec=4852 Nov 05 11:55:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7763(8168917) WEXITSTATUS 1 Nov 05 11:55:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7763(8168917) done Nov 05 11:55:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 24708: QOSMinMemory Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:33 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:34 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:35 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8187(8175112) WEXITSTATUS 1 Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8187(8175112) done Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2407(8165638) WEXITSTATUS 0 Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2407(8165638) done Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 11:55:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6489(8164405) WEXITSTATUS 1 Nov 05 11:55:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6489(8164405) done Nov 05 11:55:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:44 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175447 InitPrio=35093 usec=2908 Nov 05 11:55:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:55:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175446 NodeList=r807u03n04 #CPUs=8 Partition=day Nov 05 11:55:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175447 NodeList=r807u37n01 #CPUs=1 Partition=bigmem Nov 05 11:55:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144160 NodeList=r903u07n[03-04] #CPUs=96 Partition=scavenge Nov 05 11:55:57 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8169048 Nov 05 11:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6276(8163595) WEXITSTATUS 1 Nov 05 11:55:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6276(8163595) done Nov 05 11:55:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_30(8165386) WEXITSTATUS 0 Nov 05 11:56:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_30(8165386) done Nov 05 11:56:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154699 WEXITSTATUS 0 Nov 05 11:56:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154699 done Nov 05 11:56:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_19(8175448) in pi_hammes_schiffer on r914u13n02 Nov 05 11:56:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_20(8175449) in pi_hammes_schiffer on r914u13n03 Nov 05 11:56:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172918 in day on r806u02n01 Nov 05 11:56:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175213 in day on r815u03n05 Nov 05 11:56:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 11:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1114(7573283) WEXITSTATUS 0 Nov 05 11:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1114(7573283) done Nov 05 11:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_281(8114547) WEXITSTATUS 0 Nov 05 11:56:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_281(8114547) done Nov 05 11:56:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166591 in gpu on r908u24n01 Nov 05 11:56:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8166592 in gpu on r908u24n02 Nov 05 11:56:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7843(8169560) WEXITSTATUS 1 Nov 05 11:56:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7843(8169560) done Nov 05 11:56:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175246_0(8175246) WEXITSTATUS 0 Nov 05 11:56:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175246_0(8175246) done Nov 05 11:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135459 WEXITSTATUS 0 Nov 05 11:56:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135459 done Nov 05 11:56:38 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u32n02 Nov 05 11:56:38 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u32n01 Nov 05 11:56:38 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u30n03 Nov 05 11:56:38 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u30n04 Nov 05 11:56:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170530 WEXITSTATUS 0 Nov 05 11:56:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170530 done Nov 05 11:56:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175450 InitPrio=24888 usec=3615 Nov 05 11:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8088(8172675) WEXITSTATUS 1 Nov 05 11:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8088(8172675) done Nov 05 11:56:42 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175450 NodeList=r903u01n02 #CPUs=1 Partition=pi_zhu Nov 05 11:56:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_32(8165388) WEXITSTATUS 0 Nov 05 11:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_32(8165388) done Nov 05 11:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175450 WEXITSTATUS 0 Nov 05 11:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175450 done Nov 05 11:56:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1929(8125286) in scavenge on r907u09n02 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1508(8123768) in scavenge on r801u27n02 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2268(8126182) in scavenge on r903u34n01 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1962(8125372) in scavenge on r909u13n01 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2418(8126762) in scavenge on r914u01n03 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2692(8128063) in scavenge on r805u09n04 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2753(8128225) in scavenge on r805u11n03 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2711(8128118) in scavenge on r805u14n03 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2244(8126146) in scavenge on r912u28n04 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2858(8128498) in scavenge on r912u32n04 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2778(8128279) in scavenge on r913u28n02 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_580(8113964) in scavenge on r918u09n03 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3189(8132101) in scavenge on r918u09n03 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3431(8135271) in scavenge on r918u09n04 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2229(8126125) in scavenge on r918u32n03 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_719(7572888) in scavenge on r906u39n01 Nov 05 11:56:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_413(7572581) in scavenge on r807u37n01 Nov 05 11:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3756(8164425) WEXITSTATUS 0 Nov 05 11:56:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3756(8164425) done Nov 05 11:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2970(8175451) in scavenge on r816u23n11 Nov 05 11:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2971(8175452) in scavenge on r807u37n01 Nov 05 11:56:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:51 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175453 InitPrio=20039 usec=6092 Nov 05 11:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172514 WEXITSTATUS 0 Nov 05 11:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172514 done Nov 05 11:56:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174108 NodeList=r815u23n05 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172967 NodeList=r815u29n02 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172968 NodeList=r815u29n04 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172969 NodeList=r815u29n05 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172970 NodeList=r815u35n01 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172971 NodeList=r815u03n02 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172972 NodeList=r815u03n03 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172973 NodeList=r815u03n08 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172974 NodeList=r815u09n01 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172975 NodeList=r815u09n04 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172976 NodeList=r815u09n08 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172977 NodeList=r815u15n08 #CPUs=16 Partition=day Nov 05 11:56:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175247_0(8175247) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 11:56:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175238 uid 18734 Nov 05 11:56:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3963(8169444) WEXITSTATUS 0 Nov 05 11:56:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3963(8169444) done Nov 05 11:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_21(8175454) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:56:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175455 InitPrio=31101 usec=7744 Nov 05 11:57:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:57:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:57:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_27(8165264) WEXITSTATUS 0 Nov 05 11:57:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_27(8165264) done Nov 05 11:57:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175456 InitPrio=29089 usec=4808 Nov 05 11:57:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:18 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175457 InitPrio=20704 usec=5365 Nov 05 11:57:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7123(8165706) WEXITSTATUS 1 Nov 05 11:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7123(8165706) done Nov 05 11:57:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:28 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_709(7571674) Nov 05 11:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8139(8172902) WEXITSTATUS 1 Nov 05 11:57:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8139(8172902) done Nov 05 11:57:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175457 uid 17360 Nov 05 11:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7694(8168467) WEXITSTATUS 1 Nov 05 11:57:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7694(8168467) done Nov 05 11:57:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8163(8174003) WEXITSTATUS 1 Nov 05 11:57:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8163(8174003) done Nov 05 11:57:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_35(8165291) WEXITSTATUS 0 Nov 05 11:57:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_35(8165291) done Nov 05 11:57:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_22(8175458) NodeList=r914u09n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:57:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:00 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 11:58:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7990(8171305) WEXITSTATUS 1 Nov 05 11:58:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7990(8171305) done Nov 05 11:58:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175459 InitPrio=20704 usec=3411 Nov 05 11:58:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4022(8175206) WEXITSTATUS 0 Nov 05 11:58:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4022(8175206) done Nov 05 11:58:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175460 InitPrio=20736 usec=3470 Nov 05 11:58:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_16(8174140) WEXITSTATUS 0 Nov 05 11:58:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_16(8174140) done Nov 05 11:58:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_6(8174130) WEXITSTATUS 0 Nov 05 11:58:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_6(8174130) done Nov 05 11:58:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175446 WEXITSTATUS 0 Nov 05 11:58:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175446 done Nov 05 11:58:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_7(8174131) WEXITSTATUS 0 Nov 05 11:58:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_7(8174131) done Nov 05 11:58:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175460 NodeList=r915u29n04 #CPUs=24 Partition=pi_hammes_schiffer Nov 05 11:58:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_23(8175461) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_14(8174138) WEXITSTATUS 0 Nov 05 11:58:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_14(8174138) done Nov 05 11:58:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_24(8175462) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_8(8174132) WEXITSTATUS 0 Nov 05 11:58:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_8(8174132) done Nov 05 11:58:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_25(8175463) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_9(8174133) WEXITSTATUS 0 Nov 05 11:58:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_9(8174133) done Nov 05 11:58:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_26(8175464) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_13(8174137) WEXITSTATUS 0 Nov 05 11:58:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_13(8174137) done Nov 05 11:58:32 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_27(8175465) NodeList=r914u13n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159165 WEXITSTATUS 0 Nov 05 11:58:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159165 done Nov 05 11:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_15(8174139) WEXITSTATUS 0 Nov 05 11:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_15(8174139) done Nov 05 11:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_11(8174135) WEXITSTATUS 0 Nov 05 11:58:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_11(8174135) done Nov 05 11:58:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_28(8175466) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_29(8175467) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 11:58:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 11:58:36 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 11:58:36 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u02n03.grace.ycrc.yale.edu" Nov 05 11:58:36 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u02n03.grace.ycrc.yale.edu" Nov 05 11:58:37 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 11:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175247_0(8175247) WEXITSTATUS 0 Nov 05 11:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175247_0(8175247) done Nov 05 11:58:37 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u02n04.grace.ycrc.yale.edu" Nov 05 11:58:37 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u02n04.grace.ycrc.yale.edu" Nov 05 11:58:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:39 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u02n02.grace.ycrc.yale.edu" Nov 05 11:58:39 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u02n02.grace.ycrc.yale.edu" Nov 05 11:58:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159166 WEXITSTATUS 0 Nov 05 11:58:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159166 done Nov 05 11:58:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7979(8171127) WEXITSTATUS 1 Nov 05 11:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7979(8171127) done Nov 05 11:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_4(8174128) WEXITSTATUS 0 Nov 05 11:58:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_4(8174128) done Nov 05 11:58:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_30(8175468) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:46 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u04n01.grace.ycrc.yale.edu" Nov 05 11:58:46 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u04n01.grace.ycrc.yale.edu" Nov 05 11:58:47 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r803u02n01.grace.ycrc.yale.edu" Nov 05 11:58:47 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r803u02n01.grace.ycrc.yale.edu" Nov 05 11:58:49 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175469 NodeList=(null) usec=26055 Nov 05 11:58:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175469 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 11:58:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_3(8174127) WEXITSTATUS 0 Nov 05 11:58:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_3(8174127) done Nov 05 11:58:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_2(8174126) WEXITSTATUS 0 Nov 05 11:58:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_2(8174126) done Nov 05 11:58:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_5(8174129) WEXITSTATUS 0 Nov 05 11:58:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_5(8174129) done Nov 05 11:58:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_31(8175470) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_32(8175471) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_33(8175472) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:58:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175248_0(8175248) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 11:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1060(7572026) WEXITSTATUS 0 Nov 05 11:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_1060(7572026) done Nov 05 11:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_644(7571609) WEXITSTATUS 0 Nov 05 11:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_644(7571609) done Nov 05 11:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_919(7571885) WEXITSTATUS 0 Nov 05 11:58:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_919(7571885) done Nov 05 11:58:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_10(8174134) WEXITSTATUS 0 Nov 05 11:58:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_10(8174134) done Nov 05 11:58:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_34(8175473) NodeList=r914u09n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:58:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_26(8165382) WEXITSTATUS 0 Nov 05 11:59:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_26(8165382) done Nov 05 11:59:01 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_35(8175474) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:59:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_1(8174125) WEXITSTATUS 0 Nov 05 11:59:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_1(8174125) done Nov 05 11:59:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_36(8175475) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:59:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 11:59:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 11:59:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_12(8174136) WEXITSTATUS 0 Nov 05 11:59:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_12(8174136) done Nov 05 11:59:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_37(8175476) NodeList=r914u13n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:59:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175456 in day on r904u04n02 Nov 05 11:59:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175477 InitPrio=29089 usec=3650 Nov 05 11:59:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175478 InitPrio=29089 usec=4229 Nov 05 11:59:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175479 InitPrio=29089 usec=3857 Nov 05 11:59:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175456 WEXITSTATUS 0 Nov 05 11:59:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175456 done Nov 05 11:59:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_38(8175480) in day on r805u02n03 Nov 05 11:59:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_39(8175481) in day on r806u11n04 Nov 05 11:59:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_40(8175482) in day on r815u35n06 Nov 05 11:59:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175459 in week on r816u29n03 Nov 05 11:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_103(7572270) WEXITSTATUS 0 Nov 05 11:59:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_103(7572270) done Nov 05 11:59:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_0(8174124) WEXITSTATUS 0 Nov 05 11:59:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_0(8174124) done Nov 05 11:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_41(8175483) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 11:59:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7384(8166136) WEXITSTATUS 1 Nov 05 11:59:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7384(8166136) done Nov 05 11:59:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3718(8139334) in scavenge on r802u32n04 Nov 05 11:59:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3248(8133070) in scavenge on r912u05n04 Nov 05 11:59:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3250(8133074) in scavenge on r915u35n02 Nov 05 11:59:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3237(8132855) in scavenge on r815u15n12 Nov 05 11:59:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3467(8135369) in scavenge on r807u03n04 Nov 05 11:59:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3494(8135506) in scavenge on r909u10n01 Nov 05 11:59:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3529(8135675) in scavenge on r919u01n03 Nov 05 11:59:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_963(8115232) in scavenge on r806u28n01 Nov 05 11:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4024(8175240) WEXITSTATUS 0 Nov 05 11:59:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4024(8175240) done Nov 05 11:59:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8029(8172140) WEXITSTATUS 1 Nov 05 11:59:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8029(8172140) done Nov 05 11:59:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 11:59:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3334(8135022) Nov 05 12:00:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 12:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 12:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 12:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 12:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 12:00:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7567(8166795) WEXITSTATUS 1 Nov 05 12:00:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7567(8166795) done Nov 05 12:00:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3864(8165911) WEXITSTATUS 0 Nov 05 12:00:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3864(8165911) done Nov 05 12:00:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5819(8162781) WEXITSTATUS 1 Nov 05 12:00:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5819(8162781) done Nov 05 12:00:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6531(8164476) WEXITSTATUS 1 Nov 05 12:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6531(8164476) done Nov 05 12:00:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:22 r912u23n01 slurmctld[163760]: slurmctld: node r803u02n03 returned to service Nov 05 12:00:22 r912u23n01 slurmctld[163760]: slurmctld: node r803u02n04 returned to service Nov 05 12:00:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:23 r912u23n01 slurmctld[163760]: slurmctld: node r803u02n02 returned to service Nov 05 12:00:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:31 r912u23n01 slurmctld[163760]: slurmctld: node r803u04n01 returned to service Nov 05 12:00:31 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u30n03.grace.ycrc.yale.edu" Nov 05 12:00:31 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u30n03.grace.ycrc.yale.edu" Nov 05 12:00:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:32 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u30n04.grace.ycrc.yale.edu" Nov 05 12:00:32 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u30n04.grace.ycrc.yale.edu" Nov 05 12:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4023(8175207) WEXITSTATUS 0 Nov 05 12:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4023(8175207) done Nov 05 12:00:32 r912u23n01 slurmctld[163760]: slurmctld: node r803u02n01 returned to service Nov 05 12:00:32 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u32n02.grace.ycrc.yale.edu" Nov 05 12:00:32 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u32n02.grace.ycrc.yale.edu" Nov 05 12:00:32 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u32n01.grace.ycrc.yale.edu" Nov 05 12:00:32 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u32n01.grace.ycrc.yale.edu" Nov 05 12:00:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175248_0(8175248) WEXITSTATUS 0 Nov 05 12:00:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175248_0(8175248) done Nov 05 12:00:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175484 InitPrio=33564 usec=5723 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175485 InitPrio=33564 usec=1579 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175486 InitPrio=33564 usec=2024 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175487 InitPrio=33564 usec=1505 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175488 InitPrio=33564 usec=1696 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175489 InitPrio=33564 usec=1362 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175490 InitPrio=33564 usec=2108 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175491 InitPrio=33564 usec=1513 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175492 InitPrio=33564 usec=1785 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175493 InitPrio=33564 usec=1368 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175494 InitPrio=33564 usec=1206 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175495 InitPrio=33564 usec=1058 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175496 InitPrio=33564 usec=1079 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175497 InitPrio=33564 usec=1001 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175498 InitPrio=33564 usec=985 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175499 InitPrio=33564 usec=1051 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175500 InitPrio=33564 usec=1023 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175501 InitPrio=33564 usec=2097 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175502 InitPrio=33564 usec=1731 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175503 InitPrio=33564 usec=1164 Nov 05 12:00:38 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175504 InitPrio=33564 usec=2169 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175505 InitPrio=33564 usec=1156 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175506 InitPrio=33564 usec=1347 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175507 InitPrio=33564 usec=1135 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175508 InitPrio=33564 usec=1126 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175509 InitPrio=33564 usec=1751 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175510 InitPrio=33564 usec=1350 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175511 InitPrio=33564 usec=1194 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175512 InitPrio=33564 usec=1093 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175513 InitPrio=33564 usec=1131 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175514 InitPrio=33564 usec=1183 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175515 InitPrio=33564 usec=1221 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175516 InitPrio=33564 usec=1117 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175517 InitPrio=33564 usec=1146 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175518 InitPrio=33564 usec=1105 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175519 InitPrio=33564 usec=1130 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175520 InitPrio=33564 usec=1065 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175521 InitPrio=33564 usec=2127 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175522 InitPrio=33564 usec=1587 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175523 InitPrio=33564 usec=1229 Nov 05 12:00:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175524 InitPrio=33564 usec=1085 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175525 InitPrio=33564 usec=1785 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175526 InitPrio=33564 usec=1491 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175527 InitPrio=33564 usec=1078 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175528 InitPrio=33564 usec=1038 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175529 InitPrio=33564 usec=979 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175530 InitPrio=33564 usec=1039 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175531 InitPrio=33564 usec=1046 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175532 InitPrio=33564 usec=1013 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175533 InitPrio=33564 usec=2132 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175534 InitPrio=33564 usec=1698 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175535 InitPrio=33564 usec=1318 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175536 InitPrio=33564 usec=1187 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175537 InitPrio=33564 usec=1234 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175538 InitPrio=33564 usec=1125 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175539 InitPrio=33564 usec=1087 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175540 InitPrio=33564 usec=1159 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175541 InitPrio=33564 usec=1067 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175542 InitPrio=33564 usec=1676 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175543 InitPrio=33564 usec=1430 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175544 InitPrio=33564 usec=1088 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175545 InitPrio=33564 usec=1131 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175546 InitPrio=33564 usec=1014 Nov 05 12:00:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175547 InitPrio=33564 usec=1136 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175548 InitPrio=33564 usec=1045 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175549 InitPrio=33564 usec=1956 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175550 InitPrio=33564 usec=1474 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175551 InitPrio=33564 usec=1242 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175552 InitPrio=33564 usec=1134 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175553 InitPrio=33564 usec=1043 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175554 InitPrio=33564 usec=2053 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175555 InitPrio=33564 usec=1638 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175556 InitPrio=33564 usec=1356 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175557 InitPrio=33564 usec=1257 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175558 InitPrio=33564 usec=1214 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175559 InitPrio=33564 usec=1143 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175560 InitPrio=33564 usec=1130 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175561 InitPrio=33564 usec=1074 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175562 InitPrio=33564 usec=1225 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175563 InitPrio=33564 usec=1860 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175564 InitPrio=33564 usec=1458 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175565 InitPrio=33564 usec=1385 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175566 InitPrio=33564 usec=1146 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175567 InitPrio=33564 usec=1141 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175568 InitPrio=33564 usec=1289 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175569 InitPrio=33564 usec=1162 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175570 InitPrio=33564 usec=1101 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175571 InitPrio=33564 usec=1230 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175572 InitPrio=33564 usec=1111 Nov 05 12:00:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175573 InitPrio=33564 usec=1250 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175574 InitPrio=33564 usec=2118 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175575 InitPrio=33564 usec=1790 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175576 InitPrio=33564 usec=1841 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175577 InitPrio=33564 usec=1378 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175578 InitPrio=33564 usec=1205 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175579 InitPrio=33564 usec=1050 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175580 InitPrio=33564 usec=1961 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175581 InitPrio=33564 usec=1029 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175582 InitPrio=33564 usec=1753 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175583 InitPrio=33564 usec=1427 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175584 InitPrio=33564 usec=1133 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175585 InitPrio=33564 usec=994 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175586 InitPrio=33564 usec=1032 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175587 InitPrio=33564 usec=980 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175588 InitPrio=33564 usec=989 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175589 InitPrio=33564 usec=1048 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175590 InitPrio=33564 usec=1013 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175591 InitPrio=33564 usec=1079 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175592 InitPrio=33564 usec=1062 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175593 InitPrio=33564 usec=1045 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175594 InitPrio=33564 usec=2053 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175595 InitPrio=33564 usec=1774 Nov 05 12:00:42 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175596 InitPrio=33564 usec=1349 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175597 InitPrio=33564 usec=1932 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175598 InitPrio=33564 usec=1543 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175599 InitPrio=33564 usec=1247 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175600 InitPrio=33564 usec=1067 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175601 InitPrio=33564 usec=1198 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175602 InitPrio=33564 usec=1838 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175603 InitPrio=33564 usec=1651 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175604 InitPrio=33564 usec=1195 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175605 InitPrio=33564 usec=1067 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175606 InitPrio=33564 usec=1162 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175607 InitPrio=33564 usec=1204 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175608 InitPrio=33564 usec=1131 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175609 InitPrio=33564 usec=1198 Nov 05 12:00:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175610 InitPrio=33564 usec=1058 Nov 05 12:00:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r807u03n02 #CPUs=1 Partition=day Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r807u03n02 #CPUs=1 Partition=day Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175249_0(8175249) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175484 NodeList=r815u15n11 #CPUs=8 Partition=scavenge Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175485 NodeList=r815u23n08 #CPUs=8 Partition=scavenge Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175486 NodeList=r816u23n07 #CPUs=8 Partition=scavenge Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175487 NodeList=r816u03n10 #CPUs=8 Partition=scavenge Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175488 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175489 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:00:59 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2162(8151838) Nov 05 12:01:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:01:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:01:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172689 WEXITSTATUS 0 Nov 05 12:01:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172689 done Nov 05 12:01:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 12:01:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 12:01:13 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 12:01:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:01:20 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 12:01:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7992(8171508) WEXITSTATUS 1 Nov 05 12:01:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7992(8171508) done Nov 05 12:01:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7369(8166111) WEXITSTATUS 1 Nov 05 12:01:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7369(8166111) done Nov 05 12:01:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:01:43 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175611 InitPrio=10741 usec=9515 Nov 05 12:01:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175611 NodeList=r806u25n01 #CPUs=1 Partition=devel Nov 05 12:01:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:01:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8185(8175110) WEXITSTATUS 1 Nov 05 12:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8185(8175110) done Nov 05 12:01:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:02:15 r912u23n01 slurmctld[163760]: slurmctld: node r804u30n04 returned to service Nov 05 12:02:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:02:16 r912u23n01 slurmctld[163760]: slurmctld: node r804u30n03 returned to service Nov 05 12:02:17 r912u23n01 slurmctld[163760]: slurmctld: node r804u32n02 returned to service Nov 05 12:02:17 r912u23n01 slurmctld[163760]: slurmctld: node r804u32n01 returned to service Nov 05 12:02:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:02:22 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 12:02:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175221 in gpu on r808u14n01,r808u17n01,r903u34n01,r4513u23n01,r4513u26n01 Nov 05 12:02:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_42(8175612) in day on r815u03n03 Nov 05 12:02:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4026(8175613) in gpu on r4513u20n01 Nov 05 12:02:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4027(8175614) in gpu on r903u40n01 Nov 05 12:02:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u03n02 Nov 05 12:02:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_122(7572289) WEXITSTATUS 0 Nov 05 12:02:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_122(7572289) done Nov 05 12:02:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_638(8114907) WEXITSTATUS 0 Nov 05 12:02:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_638(8114907) done Nov 05 12:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1023(7573192) WEXITSTATUS 0 Nov 05 12:02:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1023(7573192) done Nov 05 12:02:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:02:29 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175490 in scavenge on r816u09n06 Nov 05 12:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_24(8165380) WEXITSTATUS 0 Nov 05 12:02:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_24(8165380) done Nov 05 12:02:31 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_43(8175615) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:02:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:02:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175249_0(8175249) WEXITSTATUS 0 Nov 05 12:02:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175249_0(8175249) done Nov 05 12:02:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:02:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4027(8175614) WEXITSTATUS 0 Nov 05 12:02:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4027(8175614) done Nov 05 12:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2445(8127464) in scavenge on r801u09n04 Nov 05 12:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3533(8135691) in scavenge on r802u30n03 Nov 05 12:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3649(8137586) in scavenge on r4513u29n01 Nov 05 12:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2786(8128290) in scavenge on r805u09n04 Nov 05 12:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3396(8135212) in scavenge on r919u36n03 Nov 05 12:02:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3851(8144680) in scavenge on r815u35n04 Nov 05 12:02:42 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 12:02:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:02:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:02:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175250_0(8175250) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:03:01 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_1086(7573255) Nov 05 12:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7703(8168498) WEXITSTATUS 1 Nov 05 12:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7703(8168498) done Nov 05 12:03:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:03:09 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4021(8174109) WEXITSTATUS 0 Nov 05 12:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4021(8174109) done Nov 05 12:03:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 12:03:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 12:03:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 12:03:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8078(8172586) WEXITSTATUS 1 Nov 05 12:03:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8078(8172586) done Nov 05 12:03:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2499(8165938) WEXITSTATUS 0 Nov 05 12:03:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2499(8165938) done Nov 05 12:03:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:33 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:03:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7573(8166811) WEXITSTATUS 1 Nov 05 12:03:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7573(8166811) done Nov 05 12:03:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7735(8168726) WEXITSTATUS 1 Nov 05 12:03:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7735(8168726) done Nov 05 12:03:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:03:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175489 WEXITSTATUS 0 Nov 05 12:03:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175489 done Nov 05 12:03:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175488 WEXITSTATUS 0 Nov 05 12:03:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175488 done Nov 05 12:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175484 WEXITSTATUS 0 Nov 05 12:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175484 done Nov 05 12:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175486 WEXITSTATUS 0 Nov 05 12:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175486 done Nov 05 12:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175487 WEXITSTATUS 0 Nov 05 12:03:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175487 done Nov 05 12:03:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175485 WEXITSTATUS 0 Nov 05 12:03:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175485 done Nov 05 12:03:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2293(8165156) WEXITSTATUS 0 Nov 05 12:04:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2293(8165156) done Nov 05 12:04:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:04:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_18(8165374) WEXITSTATUS 0 Nov 05 12:04:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_18(8165374) done Nov 05 12:04:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_44(8175616) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:04:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:04:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175250_0(8175250) WEXITSTATUS 0 Nov 05 12:04:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175250_0(8175250) done Nov 05 12:04:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8125(8172810) WEXITSTATUS 1 Nov 05 12:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8125(8172810) done Nov 05 12:04:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:04:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135452 WEXITSTATUS 0 Nov 05 12:04:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135452 done Nov 05 12:04:50 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n02 Nov 05 12:04:50 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n04 Nov 05 12:04:50 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n01 Nov 05 12:04:50 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n03 Nov 05 12:04:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:04:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:04:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:04:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:04:53 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:04:54 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:04:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175251_0(8175251) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175491 NodeList=r815u15n11 #CPUs=8 Partition=scavenge Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175492 NodeList=r815u23n08 #CPUs=8 Partition=scavenge Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175493 NodeList=r816u23n07 #CPUs=8 Partition=scavenge Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175494 NodeList=r816u03n10 #CPUs=8 Partition=scavenge Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175495 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175496 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_255(8114521) WEXITSTATUS 0 Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_255(8114521) done Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7005(8165399) WEXITSTATUS 1 Nov 05 12:04:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7005(8165399) done Nov 05 12:04:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:05:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7965(8170946) WEXITSTATUS 1 Nov 05 12:05:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7965(8170946) done Nov 05 12:05:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7994(8171510) WEXITSTATUS 1 Nov 05 12:05:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7994(8171510) done Nov 05 12:05:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8179(8174118) WEXITSTATUS 1 Nov 05 12:05:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8179(8174118) done Nov 05 12:05:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:21 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175617 InitPrio=25500 usec=4350 Nov 05 12:05:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_20(8165256) WEXITSTATUS 0 Nov 05 12:05:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_20(8165256) done Nov 05 12:05:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175618 InitPrio=25500 usec=3558 Nov 05 12:05:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175478 in day on r816u03n02 Nov 05 12:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_45(8175619) in day on r815u03n08 Nov 05 12:05:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_46(8175620) in day on r816u09n07 Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4028(8175621) in gpu on r4513u26n01 Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4029(8175622) in gpu on r903u40n01 Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_216(7572384) WEXITSTATUS 0 Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_216(7572384) done Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7572167_288(7572456) Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_129(8114395) Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_495(7572663) WEXITSTATUS 0 Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_495(7572663) done Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_642(8114911) WEXITSTATUS 0 Nov 05 12:05:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_642(8114911) done Nov 05 12:05:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175497 in scavenge on r816u03n09 Nov 05 12:05:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175498 in scavenge on r816u15n04 Nov 05 12:05:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175490 WEXITSTATUS 0 Nov 05 12:05:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175490 done Nov 05 12:05:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170256_1(8170285) WEXITSTATUS 0 Nov 05 12:05:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170256_1(8170285) done Nov 05 12:05:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135453 WEXITSTATUS 0 Nov 05 12:05:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135453 done Nov 05 12:05:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:46 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n04 Nov 05 12:05:46 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n03 Nov 05 12:05:46 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u11n02 Nov 05 12:05:46 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u11n01 Nov 05 12:05:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3224(8132619) in scavenge on r904u01n01 Nov 05 12:05:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3537(8135783) in scavenge on r808u08n01 Nov 05 12:05:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_720(8114201) in scavenge on r4513u23n01 Nov 05 12:05:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3541(8135808) in scavenge on r807u03n02 Nov 05 12:05:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3703(8138810) in scavenge on r919u09n01 Nov 05 12:05:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:05:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8170256_4(8170256) NodeList=r4512u32n01 #CPUs=1 Partition=transfer Nov 05 12:05:57 r912u23n01 slurmctld[163760]: slurmctld: _job_create: exceeded association/QOS limit for user 23079: QOSMaxSubmitJobPerUserLimit Nov 05 12:05:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits) Nov 05 12:06:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2972(8175623) in scavenge on r808u08n01 Nov 05 12:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_39(8165407) WEXITSTATUS 0 Nov 05 12:06:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_39(8165407) done Nov 05 12:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_47(8175624) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:06:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:06:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8170284 uid 23079 Nov 05 12:06:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:06:36 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175625 InitPrio=13783 usec=3051 Nov 05 12:06:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175251_0(8175251) WEXITSTATUS 0 Nov 05 12:06:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175251_0(8175251) done Nov 05 12:06:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172918 WEXITSTATUS 1 Nov 05 12:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172918 done Nov 05 12:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7636(8167091) WEXITSTATUS 1 Nov 05 12:06:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7636(8167091) done Nov 05 12:06:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:06:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7991(8171306) WEXITSTATUS 1 Nov 05 12:06:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7991(8171306) done Nov 05 12:06:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:06:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:06:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175455 NodeList=r806u02n01 #CPUs=12 Partition=day Nov 05 12:06:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175252_0(8175252) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:06:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175625 NodeList=r915u01n01 #CPUs=4 Partition=gpu_devel Nov 05 12:07:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175626 InitPrio=59397 usec=5524 Nov 05 12:07:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:07:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2288(8165137) WEXITSTATUS 0 Nov 05 12:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2288(8165137) done Nov 05 12:07:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171121 WEXITSTATUS 0 Nov 05 12:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8171121 done Nov 05 12:07:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:07:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_36(8165404) WEXITSTATUS 0 Nov 05 12:07:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_36(8165404) done Nov 05 12:07:28 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8174123_48(8175627) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:07:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2257(8165069) WEXITSTATUS 0 Nov 05 12:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2257(8165069) done Nov 05 12:07:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7946(8170647) WEXITSTATUS 1 Nov 05 12:07:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7946(8170647) done Nov 05 12:07:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:07:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:07:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175626 NodeList=r806u09n04 #CPUs=8 Partition=day Nov 05 12:07:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175477 NodeList=r816u09n05 #CPUs=16 Partition=day Nov 05 12:07:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_817(8115086) WEXITSTATUS 0 Nov 05 12:07:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_817(8115086) done Nov 05 12:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175495 WEXITSTATUS 0 Nov 05 12:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175495 done Nov 05 12:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175496 WEXITSTATUS 0 Nov 05 12:07:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175496 done Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175494 WEXITSTATUS 0 Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175494 done Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175491 WEXITSTATUS 0 Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175491 done Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175493 WEXITSTATUS 0 Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175493 done Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175492 WEXITSTATUS 0 Nov 05 12:07:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175492 done Nov 05 12:08:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:08:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4010(8172665) WEXITSTATUS 0 Nov 05 12:08:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4010(8172665) done Nov 05 12:08:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:08:10 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n03 returned to service Nov 05 12:08:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:08:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_23(8165259) WEXITSTATUS 0 Nov 05 12:08:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_23(8165259) done Nov 05 12:08:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:08:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172979 in day on r816u03n03 Nov 05 12:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7920(8170363) WEXITSTATUS 1 Nov 05 12:08:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7920(8170363) done Nov 05 12:08:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8174123_49(8174123) in day on r805u06n01 Nov 05 12:08:38 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n04.grace.ycrc.yale.edu" Nov 05 12:08:38 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n04.grace.ycrc.yale.edu" Nov 05 12:08:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:08:38 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n02.grace.ycrc.yale.edu" Nov 05 12:08:38 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n02.grace.ycrc.yale.edu" Nov 05 12:08:39 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 12:08:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4030(8175628) in gpu on r808u14n01 Nov 05 12:08:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175214 in day on r815u35n01 Nov 05 12:08:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u03n02 Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175252_0(8175252) WEXITSTATUS 0 Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175252_0(8175252) done Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175497 WEXITSTATUS 0 Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175497 done Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175498 WEXITSTATUS 0 Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175498 done Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_649(8114918) WEXITSTATUS 0 Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_649(8114918) done Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_231(8114497) WEXITSTATUS 0 Nov 05 12:08:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_231(8114497) done Nov 05 12:08:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175499 in scavenge on r815u15n11 Nov 05 12:08:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175500 in scavenge on r815u23n08 Nov 05 12:08:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175501 in scavenge on r816u23n07 Nov 05 12:08:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175502 in scavenge on r816u03n10 Nov 05 12:08:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175503 in scavenge on r816u03n11 Nov 05 12:08:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175504 in scavenge on r816u03n11 Nov 05 12:08:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175505 in scavenge on r816u09n05 Nov 05 12:08:41 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175506 in scavenge on r816u09n06 Nov 05 12:08:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:08:44 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n01.grace.ycrc.yale.edu" Nov 05 12:08:44 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n01.grace.ycrc.yale.edu" Nov 05 12:08:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8032(8172278) WEXITSTATUS 1 Nov 05 12:08:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8032(8172278) done Nov 05 12:08:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:08:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2610(8127858) in scavenge on r903u01n01 Nov 05 12:08:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3709(8138924) in scavenge on r916u04n01 Nov 05 12:08:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3234(8132746) in scavenge on r805u06n01 Nov 05 12:08:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2649(8127931) in scavenge on r906u41n04 Nov 05 12:08:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3611(8136976) in scavenge on r904u04n02 Nov 05 12:08:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3488(8135481) in scavenge on r815u35n01 Nov 05 12:08:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_707(8114976) in scavenge on r802u02n02 Nov 05 12:08:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:08:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175253_0(8175253) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:08:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175507 NodeList=r816u03n09 #CPUs=8 Partition=scavenge Nov 05 12:08:57 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175508 NodeList=r816u15n09 #CPUs=8 Partition=scavenge Nov 05 12:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7978(8171126) WEXITSTATUS 1 Nov 05 12:09:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7978(8171126) done Nov 05 12:09:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:03 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3358(8135127) Nov 05 12:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7776(8169115) WEXITSTATUS 1 Nov 05 12:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7776(8169115) done Nov 05 12:09:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7819(8169456) WEXITSTATUS 1 Nov 05 12:09:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7819(8169456) done Nov 05 12:09:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5800(8162759) WEXITSTATUS 1 Nov 05 12:09:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5800(8162759) done Nov 05 12:09:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:09:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:09:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175629 InitPrio=31343 usec=8989 Nov 05 12:09:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:15 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175630 NodeList=(null) usec=19128 Nov 05 12:09:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175630 NodeList=r806u25n01 #CPUs=4 Partition=devel Nov 05 12:09:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:30 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u11n01.grace.ycrc.yale.edu" Nov 05 12:09:30 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u11n01.grace.ycrc.yale.edu" Nov 05 12:09:31 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n04.grace.ycrc.yale.edu" Nov 05 12:09:31 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n04.grace.ycrc.yale.edu" Nov 05 12:09:31 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u11n02.grace.ycrc.yale.edu" Nov 05 12:09:31 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u11n02.grace.ycrc.yale.edu" Nov 05 12:09:32 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n03.grace.ycrc.yale.edu" Nov 05 12:09:32 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n03.grace.ycrc.yale.edu" Nov 05 12:09:33 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3831(8144445) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:09:35 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3831(8144445) Nov 05 12:09:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8081(8172668) WEXITSTATUS 1 Nov 05 12:09:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8081(8172668) done Nov 05 12:09:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_42(8165298) WEXITSTATUS 0 Nov 05 12:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_42(8165298) done Nov 05 12:09:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:49 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175631 NodeList=(null) usec=26902 Nov 05 12:09:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175631 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 12:09:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_40(8175482) WEXITSTATUS 0 Nov 05 12:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_40(8175482) done Nov 05 12:09:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175629 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 12:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175479 NodeList=r816u09n02 #CPUs=16 Partition=day Nov 05 12:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175509 NodeList=r816u23n06 #CPUs=8 Partition=scavenge Nov 05 12:09:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175510 NodeList=r816u29n02 #CPUs=8 Partition=scavenge Nov 05 12:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8009(8171683) WEXITSTATUS 1 Nov 05 12:09:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8009(8171683) done Nov 05 12:09:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:04 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3359(8135130) Nov 05 12:10:05 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:10:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_26(8175464) WEXITSTATUS 0 Nov 05 12:10:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_26(8175464) done Nov 05 12:10:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8098(8172739) WEXITSTATUS 1 Nov 05 12:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8098(8172739) done Nov 05 12:10:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:22 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n04 returned to service Nov 05 12:10:23 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n02 returned to service Nov 05 12:10:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_29(8175467) WEXITSTATUS 0 Nov 05 12:10:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_29(8175467) done Nov 05 12:10:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:28 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n01 returned to service Nov 05 12:10:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_23(8175461) WEXITSTATUS 0 Nov 05 12:10:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_23(8175461) done Nov 05 12:10:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175253_0(8175253) WEXITSTATUS 0 Nov 05 12:10:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175253_0(8175253) done Nov 05 12:10:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_31(8175470) WEXITSTATUS 0 Nov 05 12:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_31(8175470) done Nov 05 12:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_24(8175462) WEXITSTATUS 0 Nov 05 12:10:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_24(8175462) done Nov 05 12:10:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_32(8175471) WEXITSTATUS 0 Nov 05 12:10:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_32(8175471) done Nov 05 12:10:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7945(8170646) WEXITSTATUS 1 Nov 05 12:10:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7945(8170646) done Nov 05 12:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8155(8172960) WEXITSTATUS 1 Nov 05 12:10:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8155(8172960) done Nov 05 12:10:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_27(8175465) WEXITSTATUS 0 Nov 05 12:10:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_27(8175465) done Nov 05 12:10:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_25(8175463) WEXITSTATUS 0 Nov 05 12:10:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_25(8175463) done Nov 05 12:10:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:10:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175254_0(8175254) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:10:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175511 NodeList=r815u09n09 #CPUs=8 Partition=scavenge Nov 05 12:10:57 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 12:11:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_30(8175468) WEXITSTATUS 0 Nov 05 12:11:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_30(8175468) done Nov 05 12:11:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_33(8175472) WEXITSTATUS 0 Nov 05 12:11:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_33(8175472) done Nov 05 12:11:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_36(8175475) WEXITSTATUS 0 Nov 05 12:11:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_36(8175475) done Nov 05 12:11:15 r912u23n01 slurmctld[163760]: slurmctld: node r804u11n01 returned to service Nov 05 12:11:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:16 r912u23n01 slurmctld[163760]: slurmctld: node r804u11n02 returned to service Nov 05 12:11:16 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n04 returned to service Nov 05 12:11:17 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n03 returned to service Nov 05 12:11:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2260(8165080) WEXITSTATUS 0 Nov 05 12:11:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2260(8165080) done Nov 05 12:11:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2470(8165781) WEXITSTATUS 0 Nov 05 12:11:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2470(8165781) done Nov 05 12:11:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:34 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2394(8152438) Nov 05 12:11:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175218 in day on r815u09n08 Nov 05 12:11:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175242 in day on r815u35n06 Nov 05 12:11:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175512 in scavenge on r816u15n04 Nov 05 12:11:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_34(8175473) WEXITSTATUS 0 Nov 05 12:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_34(8175473) done Nov 05 12:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175503 WEXITSTATUS 0 Nov 05 12:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175503 done Nov 05 12:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175504 WEXITSTATUS 0 Nov 05 12:11:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175504 done Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_28(8175466) WEXITSTATUS 0 Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_28(8175466) done Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175502 WEXITSTATUS 0 Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175502 done Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175499 WEXITSTATUS 0 Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175499 done Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175501 WEXITSTATUS 0 Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175501 done Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175500 WEXITSTATUS 0 Nov 05 12:11:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175500 done Nov 05 12:11:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3646(8137583) in scavenge on r801u30n03 Nov 05 12:11:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3657(8137917) in scavenge on r801u09n03 Nov 05 12:11:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3852(8144681) in scavenge on r911u01n01 Nov 05 12:11:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3853(8144682) in scavenge on r802u06n01 Nov 05 12:11:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1122(8123233) in scavenge on r802u09n01 Nov 05 12:11:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3736(8141693) in scavenge on r815u03n05 Nov 05 12:11:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3738(8141698) in scavenge on r815u35n06 Nov 05 12:11:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2433(8165682) WEXITSTATUS 0 Nov 05 12:11:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2433(8165682) done Nov 05 12:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175505 WEXITSTATUS 0 Nov 05 12:11:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175505 done Nov 05 12:11:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_37(8175476) WEXITSTATUS 0 Nov 05 12:11:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_37(8175476) done Nov 05 12:11:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175506 WEXITSTATUS 0 Nov 05 12:11:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175506 done Nov 05 12:11:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172978 NodeList=r815u09n02 #CPUs=16 Partition=day Nov 05 12:11:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175513 NodeList=r815u15n11 #CPUs=8 Partition=scavenge Nov 05 12:11:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175514 NodeList=r815u23n08 #CPUs=8 Partition=scavenge Nov 05 12:11:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175515 NodeList=r816u23n07 #CPUs=8 Partition=scavenge Nov 05 12:11:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175516 NodeList=r816u03n10 #CPUs=8 Partition=scavenge Nov 05 12:11:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175517 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:11:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175518 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:11:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175507 WEXITSTATUS 0 Nov 05 12:11:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175507 done Nov 05 12:11:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175508 WEXITSTATUS 0 Nov 05 12:11:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175508 done Nov 05 12:12:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2318(8165272) WEXITSTATUS 0 Nov 05 12:12:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2318(8165272) done Nov 05 12:12:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_47(8170587) WEXITSTATUS 0 Nov 05 12:12:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_47(8170587) done Nov 05 12:12:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2254(8165060) WEXITSTATUS 0 Nov 05 12:12:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2254(8165060) done Nov 05 12:12:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8028(8172139) WEXITSTATUS 1 Nov 05 12:12:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8028(8172139) done Nov 05 12:12:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175632 InitPrio=17452 usec=3131 Nov 05 12:12:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175632 NodeList=r801u04n04 #CPUs=4 Partition=pi_esi Nov 05 12:12:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_53(8170593) WEXITSTATUS 0 Nov 05 12:12:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_53(8170593) done Nov 05 12:12:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7268(8165934) WEXITSTATUS 1 Nov 05 12:12:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7268(8165934) done Nov 05 12:12:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175254_0(8175254) WEXITSTATUS 0 Nov 05 12:12:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175254_0(8175254) done Nov 05 12:12:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_61(8170601) WEXITSTATUS 0 Nov 05 12:12:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_61(8170601) done Nov 05 12:12:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159164 WEXITSTATUS 0 Nov 05 12:12:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159164 done Nov 05 12:12:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_60(8170600) WEXITSTATUS 0 Nov 05 12:12:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_60(8170600) done Nov 05 12:12:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175479 WEXITSTATUS 1 Nov 05 12:12:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175479 done Nov 05 12:12:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_64(8170604) WEXITSTATUS 0 Nov 05 12:12:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_64(8170604) done Nov 05 12:12:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172980 NodeList=r816u09n02 #CPUs=16 Partition=day Nov 05 12:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175255_0(8175255) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175519 NodeList=r816u03n09 #CPUs=8 Partition=scavenge Nov 05 12:12:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175520 NodeList=r816u15n09 #CPUs=8 Partition=scavenge Nov 05 12:12:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8122(8172807) WEXITSTATUS 1 Nov 05 12:12:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8122(8172807) done Nov 05 12:12:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:12:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175509 WEXITSTATUS 0 Nov 05 12:12:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175509 done Nov 05 12:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175510 WEXITSTATUS 0 Nov 05 12:13:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175510 done Nov 05 12:13:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:13:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:13:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8067(8172575) WEXITSTATUS 1 Nov 05 12:13:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8067(8172575) done Nov 05 12:13:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_48(8170588) WEXITSTATUS 0 Nov 05 12:13:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_48(8170588) done Nov 05 12:13:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175629 WEXITSTATUS 0 Nov 05 12:13:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175629 done Nov 05 12:13:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_21(8165377) WEXITSTATUS 0 Nov 05 12:13:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_21(8165377) done Nov 05 12:13:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2404(8165625) WEXITSTATUS 0 Nov 05 12:13:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2404(8165625) done Nov 05 12:13:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4026(8175613) WEXITSTATUS 0 Nov 05 12:13:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4026(8175613) done Nov 05 12:13:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_41(8175483) WEXITSTATUS 0 Nov 05 12:13:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_41(8175483) done Nov 05 12:13:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_55(8170595) WEXITSTATUS 0 Nov 05 12:13:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_55(8170595) done Nov 05 12:13:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7880(8169912) WEXITSTATUS 1 Nov 05 12:13:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7880(8169912) done Nov 05 12:13:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8039(8172420) WEXITSTATUS 1 Nov 05 12:13:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8039(8172420) done Nov 05 12:13:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175521 NodeList=r815u09n10 #CPUs=8 Partition=scavenge Nov 05 12:13:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175522 NodeList=r816u09n09 #CPUs=8 Partition=scavenge Nov 05 12:13:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175523 NodeList=r816u23n11 #CPUs=8 Partition=scavenge Nov 05 12:13:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175524 NodeList=r816u23n06 #CPUs=8 Partition=scavenge Nov 05 12:13:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175525 NodeList=r816u29n02 #CPUs=8 Partition=scavenge Nov 05 12:13:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_66(8170606) WEXITSTATUS 0 Nov 05 12:13:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_66(8170606) done Nov 05 12:13:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175511 WEXITSTATUS 0 Nov 05 12:13:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175511 done Nov 05 12:13:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_57(8170597) WEXITSTATUS 0 Nov 05 12:13:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_57(8170597) done Nov 05 12:14:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3703(8138810) WEXITSTATUS 1 Nov 05 12:14:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3703(8138810) done Nov 05 12:14:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2265(8165085) WEXITSTATUS 0 Nov 05 12:14:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2265(8165085) done Nov 05 12:14:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_67(8170607) WEXITSTATUS 0 Nov 05 12:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_67(8170607) done Nov 05 12:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_49(8170589) WEXITSTATUS 0 Nov 05 12:14:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_49(8170589) done Nov 05 12:14:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_54(8170594) WEXITSTATUS 0 Nov 05 12:14:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_54(8170594) done Nov 05 12:14:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1623(8124198) WEXITSTATUS 1 Nov 05 12:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_1623(8124198) done Nov 05 12:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_56(8170596) WEXITSTATUS 0 Nov 05 12:14:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_56(8170596) done Nov 05 12:14:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_46(8170586) WEXITSTATUS 0 Nov 05 12:14:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_46(8170586) done Nov 05 12:14:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7627(8167072) WEXITSTATUS 1 Nov 05 12:14:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7627(8167072) done Nov 05 12:14:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:26 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172983 in day on r816u09n08 Nov 05 12:14:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_62(8170602) WEXITSTATUS 0 Nov 05 12:14:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_62(8170602) done Nov 05 12:14:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4031(8175633) in gpu on r4513u20n01 Nov 05 12:14:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u03n02 Nov 05 12:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_50(8170590) WEXITSTATUS 0 Nov 05 12:14:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_50(8170590) done Nov 05 12:14:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175526 in scavenge on r815u09n09 Nov 05 12:14:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175527 in scavenge on r815u29n02 Nov 05 12:14:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175528 in scavenge on r815u35n03 Nov 05 12:14:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175529 in scavenge on r815u35n03 Nov 05 12:14:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175530 in scavenge on r816u09n05 Nov 05 12:14:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175531 in scavenge on r816u09n06 Nov 05 12:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4013(8172734) WEXITSTATUS 0 Nov 05 12:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4013(8172734) done Nov 05 12:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_65(8170605) WEXITSTATUS 0 Nov 05 12:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_65(8170605) done Nov 05 12:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175255_0(8175255) WEXITSTATUS 0 Nov 05 12:14:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175255_0(8175255) done Nov 05 12:14:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175478 WEXITSTATUS 0 Nov 05 12:14:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175478 done Nov 05 12:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7476(8166433) WEXITSTATUS 1 Nov 05 12:14:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7476(8166433) done Nov 05 12:14:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175512 WEXITSTATUS 0 Nov 05 12:14:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175512 done Nov 05 12:14:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3769(8143157) in scavenge on r802u11n03 Nov 05 12:14:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2453(8127479) in scavenge on r806u09n03 Nov 05 12:14:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2324(8126291) in scavenge on r919u09n01 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175274 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175275 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175277 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175279 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175281 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175283 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175284 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175278 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175276 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175280 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175282 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175285 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175288 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175289 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175290 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175291 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175292 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175293 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175286 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175294 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175287 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175295 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175296 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175298 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175299 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175300 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175301 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175297 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175302 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175303 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175305 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175304 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175306 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175307 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175308 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175309 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175310 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175311 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175312 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175313 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175315 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175314 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175316 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175317 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175318 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175319 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175320 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175321 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175322 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175323 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175324 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175325 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175326 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175327 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175328 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175329 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175330 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175331 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175332 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175333 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175334 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175335 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175336 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175337 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175338 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175339 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175340 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175341 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175342 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175343 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175344 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175345 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175346 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175347 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175348 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175349 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175350 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175351 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175353 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175352 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175354 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175355 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175356 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175357 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175358 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175359 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175360 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175361 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175362 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175363 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175364 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175365 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175366 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175367 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175368 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175369 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175370 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175371 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175372 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175373 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175375 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175374 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175376 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175378 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175377 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175379 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175380 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175381 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175382 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175383 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175384 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175385 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175386 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175387 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175388 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175389 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175390 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175391 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175392 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175393 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175394 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175395 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175396 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175397 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175398 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175399 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175400 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175401 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175402 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175403 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175404 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175405 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175406 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175407 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175408 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175409 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175410 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175411 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175412 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175413 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175414 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175415 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175416 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175417 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175418 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175419 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175420 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175421 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175422 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175423 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175424 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175425 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175426 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175427 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175428 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175429 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175430 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175431 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175432 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175433 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175434 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175435 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175436 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175437 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175438 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175439 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175440 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175441 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175442 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175443 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175444 uid 16603 Nov 05 12:14:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175445 uid 16603 Nov 05 12:14:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7848(8169617) WEXITSTATUS 1 Nov 05 12:14:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7848(8169617) done Nov 05 12:14:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:14:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172981 NodeList=r816u03n02 #CPUs=16 Partition=day Nov 05 12:14:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175256_0(8175256) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:14:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175532 NodeList=r815u09n11 #CPUs=8 Partition=scavenge Nov 05 12:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175517 WEXITSTATUS 0 Nov 05 12:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175517 done Nov 05 12:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175518 WEXITSTATUS 0 Nov 05 12:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175518 done Nov 05 12:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175516 WEXITSTATUS 0 Nov 05 12:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175516 done Nov 05 12:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175513 WEXITSTATUS 0 Nov 05 12:14:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175513 done Nov 05 12:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175515 WEXITSTATUS 0 Nov 05 12:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175515 done Nov 05 12:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175514 WEXITSTATUS 0 Nov 05 12:14:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175514 done Nov 05 12:15:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_51(8170591) WEXITSTATUS 0 Nov 05 12:15:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_51(8170591) done Nov 05 12:15:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:05 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2438(8152596) Nov 05 12:15:05 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2470(8152628) Nov 05 12:15:05 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2511(8152669) Nov 05 12:15:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_110(8170694) WEXITSTATUS 0 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_110(8170694) done Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175477 WEXITSTATUS 0 Nov 05 12:15:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175477 done Nov 05 12:15:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_59(8170599) WEXITSTATUS 0 Nov 05 12:15:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_59(8170599) done Nov 05 12:15:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_588(8171493) WEXITSTATUS 0 Nov 05 12:15:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_588(8171493) done Nov 05 12:15:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8011(8171812) WEXITSTATUS 1 Nov 05 12:15:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8011(8171812) done Nov 05 12:15:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1512(8143571) WEXITSTATUS 0 Nov 05 12:15:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1512(8143571) done Nov 05 12:15:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_63(8170603) WEXITSTATUS 0 Nov 05 12:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_63(8170603) done Nov 05 12:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7980(8171128) WEXITSTATUS 1 Nov 05 12:15:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7980(8171128) done Nov 05 12:15:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6940(8165223) WEXITSTATUS 1 Nov 05 12:15:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6940(8165223) done Nov 05 12:15:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8130(8172837) WEXITSTATUS 1 Nov 05 12:15:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8130(8172837) done Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175256 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175258 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175257 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175260 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175264 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175262 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175259 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175263 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175265 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175261 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175268 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175269 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175270 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175266 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175267 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175271 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175272 uid 16603 Nov 05 12:15:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175273 uid 16603 Nov 05 12:15:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:37 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_52(8170592) WEXITSTATUS 0 Nov 05 12:15:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_52(8170592) done Nov 05 12:15:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2271(8165102) WEXITSTATUS 0 Nov 05 12:15:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2271(8165102) done Nov 05 12:15:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_114(8170698) WEXITSTATUS 0 Nov 05 12:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_114(8170698) done Nov 05 12:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_38(8170578) WEXITSTATUS 0 Nov 05 12:15:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_38(8170578) done Nov 05 12:15:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_119(8170703) WEXITSTATUS 0 Nov 05 12:15:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_119(8170703) done Nov 05 12:15:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5675(8162167) WEXITSTATUS 1 Nov 05 12:15:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5675(8162167) done Nov 05 12:15:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172982 NodeList=r816u09n05 #CPUs=16 Partition=day Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175533 NodeList=r815u15n11 #CPUs=8 Partition=scavenge Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175534 NodeList=r815u23n08 #CPUs=8 Partition=scavenge Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175535 NodeList=r816u03n10 #CPUs=8 Partition=scavenge Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175536 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175537 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175538 NodeList=r816u23n07 #CPUs=8 Partition=scavenge Nov 05 12:15:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175539 NodeList=r816u23n09 #CPUs=8 Partition=scavenge Nov 05 12:15:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175519 WEXITSTATUS 0 Nov 05 12:15:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175519 done Nov 05 12:15:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:15:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175520 WEXITSTATUS 0 Nov 05 12:15:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175520 done Nov 05 12:16:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_116(8170700) WEXITSTATUS 0 Nov 05 12:16:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_116(8170700) done Nov 05 12:16:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175634 InitPrio=21331 usec=6620 Nov 05 12:16:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_8(8170548) WEXITSTATUS 0 Nov 05 12:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_8(8170548) done Nov 05 12:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_5(8170545) WEXITSTATUS 0 Nov 05 12:16:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_5(8170545) done Nov 05 12:16:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169564 WEXITSTATUS 0 Nov 05 12:16:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169564 done Nov 05 12:16:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_594(8171499) WEXITSTATUS 0 Nov 05 12:16:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_594(8171499) done Nov 05 12:16:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175459 WEXITSTATUS 0 Nov 05 12:16:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175459 done Nov 05 12:16:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_580(8171485) WEXITSTATUS 0 Nov 05 12:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_580(8171485) done Nov 05 12:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_8(8165228) WEXITSTATUS 0 Nov 05 12:16:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_8(8165228) done Nov 05 12:16:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_41(8170581) WEXITSTATUS 0 Nov 05 12:16:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_41(8170581) done Nov 05 12:16:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_28(8170568) WEXITSTATUS 0 Nov 05 12:16:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_28(8170568) done Nov 05 12:16:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_90(8170630) WEXITSTATUS 0 Nov 05 12:16:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_90(8170630) done Nov 05 12:16:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_36(8170576) WEXITSTATUS 0 Nov 05 12:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_36(8170576) done Nov 05 12:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_596(8171501) WEXITSTATUS 0 Nov 05 12:16:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_596(8171501) done Nov 05 12:16:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_45(8175619) WEXITSTATUS 0 Nov 05 12:16:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_45(8175619) done Nov 05 12:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_673(8171646) WEXITSTATUS 0 Nov 05 12:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_673(8171646) done Nov 05 12:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169567 WEXITSTATUS 0 Nov 05 12:16:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169567 done Nov 05 12:16:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_9(8170549) WEXITSTATUS 0 Nov 05 12:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_9(8170549) done Nov 05 12:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_39(8170579) WEXITSTATUS 0 Nov 05 12:16:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_39(8170579) done Nov 05 12:16:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_29(8170569) WEXITSTATUS 0 Nov 05 12:16:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_29(8170569) done Nov 05 12:16:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_25(8165261) WEXITSTATUS 0 Nov 05 12:16:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165203_25(8165261) done Nov 05 12:16:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175635 InitPrio=17501 usec=4435 Nov 05 12:16:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_683(8171656) WEXITSTATUS 0 Nov 05 12:16:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_683(8171656) done Nov 05 12:16:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 12:16:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 12:16:52 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 12:16:53 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1200 Nov 05 12:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_87(8170627) WEXITSTATUS 0 Nov 05 12:16:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_87(8170627) done Nov 05 12:16:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172984 NodeList=r816u09n03 #CPUs=16 Partition=day Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175634 NodeList=r903u37n01 #CPUs=1 Partition=gpu_devel Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175635_0(8175636) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175540 NodeList=r816u03n09 #CPUs=8 Partition=scavenge Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175541 NodeList=r816u15n09 #CPUs=8 Partition=scavenge Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175542 NodeList=r816u29n03 #CPUs=8 Partition=scavenge Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175543 NodeList=r816u29n03 #CPUs=8 Partition=scavenge Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175544 NodeList=r816u29n03 #CPUs=8 Partition=scavenge Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_906(7573075) WEXITSTATUS 0 Nov 05 12:16:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_906(7573075) done Nov 05 12:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175522 WEXITSTATUS 0 Nov 05 12:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175522 done Nov 05 12:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175523 WEXITSTATUS 0 Nov 05 12:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175523 done Nov 05 12:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175521 WEXITSTATUS 0 Nov 05 12:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175521 done Nov 05 12:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175524 WEXITSTATUS 0 Nov 05 12:16:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175524 done Nov 05 12:16:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:16:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_37(8170577) WEXITSTATUS 0 Nov 05 12:16:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_37(8170577) done Nov 05 12:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_0(8175636) WEXITSTATUS 0 Nov 05 12:17:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_0(8175636) done Nov 05 12:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175525 WEXITSTATUS 0 Nov 05 12:17:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175525 done Nov 05 12:17:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8175637 NodeList=(null) usec=30696 Nov 05 12:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_109(8170693) WEXITSTATUS 0 Nov 05 12:17:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_109(8170693) done Nov 05 12:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_111(8170695) WEXITSTATUS 0 Nov 05 12:17:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_111(8170695) done Nov 05 12:17:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_30(8170570) WEXITSTATUS 0 Nov 05 12:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_30(8170570) done Nov 05 12:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_587(8171492) WEXITSTATUS 0 Nov 05 12:17:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_587(8171492) done Nov 05 12:17:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:17:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:17:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7870(8169789) WEXITSTATUS 1 Nov 05 12:17:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7870(8169789) done Nov 05 12:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_13(8170553) WEXITSTATUS 0 Nov 05 12:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_13(8170553) done Nov 05 12:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_27(8170567) WEXITSTATUS 0 Nov 05 12:17:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_27(8170567) done Nov 05 12:17:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_112(8170696) WEXITSTATUS 0 Nov 05 12:17:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_112(8170696) done Nov 05 12:17:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_107(8170691) WEXITSTATUS 0 Nov 05 12:17:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_107(8170691) done Nov 05 12:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_31(8170571) WEXITSTATUS 0 Nov 05 12:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_31(8170571) done Nov 05 12:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3992(8171507) WEXITSTATUS 0 Nov 05 12:17:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3992(8171507) done Nov 05 12:17:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_18(8170558) WEXITSTATUS 0 Nov 05 12:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_18(8170558) done Nov 05 12:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_586(8171491) WEXITSTATUS 0 Nov 05 12:17:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_586(8171491) done Nov 05 12:17:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_595(8171500) WEXITSTATUS 0 Nov 05 12:17:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_595(8171500) done Nov 05 12:17:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:23 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8165203 uid 17360 Nov 05 12:17:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_58(8170598) WEXITSTATUS 0 Nov 05 12:17:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_58(8170598) done Nov 05 12:17:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175638 InitPrio=21146 usec=4425 Nov 05 12:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_679(8171652) WEXITSTATUS 0 Nov 05 12:17:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_679(8171652) done Nov 05 12:17:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175637 in day on r904u04n03 Nov 05 12:17:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172985 in day on r807u08n04 Nov 05 12:17:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172986 in day on r918u36n03 Nov 05 12:17:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172987 in day on r919u09n01 Nov 05 12:17:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172990 in day on r815u03n02 Nov 05 12:17:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172991 in day on r815u09n06 Nov 05 12:17:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172992 in day on r815u23n01 Nov 05 12:17:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172993 in day on r815u23n02 Nov 05 12:17:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_886(7573055) WEXITSTATUS 0 Nov 05 12:17:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_886(7573055) done Nov 05 12:17:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175638 in mpi on r804u06n[03-04],r804u09n[01-04],r804u11n[01-02] Nov 05 12:17:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175635_1(8175639) in bigmem on r807u39n01 Nov 05 12:17:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4032(8175640) in gpu on r903u34n01 Nov 05 12:17:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4033(8175641) in gpu on r907u32n01 Nov 05 12:17:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175545 in scavenge on r815u09n10 Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175546 in scavenge on r816u09n09 Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175547 in scavenge on r816u23n11 Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175548 in scavenge on r815u15n08 Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175549 in scavenge on r815u15n08 Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175550 in scavenge on r815u29n04 Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175551 in scavenge on r816u15n04 Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175552 in scavenge on r816u23n06 Nov 05 12:17:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175553 in scavenge on r816u29n02 Nov 05 12:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_1(8175639) WEXITSTATUS 0 Nov 05 12:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_1(8175639) done Nov 05 12:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_639(8171612) WEXITSTATUS 0 Nov 05 12:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_639(8171612) done Nov 05 12:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_108(8170692) WEXITSTATUS 0 Nov 05 12:17:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_108(8170692) done Nov 05 12:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175526 WEXITSTATUS 0 Nov 05 12:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175526 done Nov 05 12:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_684(8171657) WEXITSTATUS 0 Nov 05 12:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_684(8171657) done Nov 05 12:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_579(8171484) WEXITSTATUS 0 Nov 05 12:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_579(8171484) done Nov 05 12:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4029(8175622) WEXITSTATUS 0 Nov 05 12:17:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4029(8175622) done Nov 05 12:17:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_597(8171502) WEXITSTATUS 0 Nov 05 12:17:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_597(8171502) done Nov 05 12:17:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175530 WEXITSTATUS 0 Nov 05 12:17:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175530 done Nov 05 12:17:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175527 WEXITSTATUS 0 Nov 05 12:17:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175527 done Nov 05 12:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_670(8171643) WEXITSTATUS 0 Nov 05 12:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_670(8171643) done Nov 05 12:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175528 WEXITSTATUS 0 Nov 05 12:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175528 done Nov 05 12:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175531 WEXITSTATUS 0 Nov 05 12:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175531 done Nov 05 12:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175529 WEXITSTATUS 0 Nov 05 12:17:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175529 done Nov 05 12:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_98(8170638) WEXITSTATUS 0 Nov 05 12:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_98(8170638) done Nov 05 12:17:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_97(8170637) WEXITSTATUS 0 Nov 05 12:17:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_97(8170637) done Nov 05 12:17:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_6(8170546) WEXITSTATUS 0 Nov 05 12:17:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_6(8170546) done Nov 05 12:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_12(8165360) WEXITSTATUS 0 Nov 05 12:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_12(8165360) done Nov 05 12:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2310(8165212) WEXITSTATUS 0 Nov 05 12:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2310(8165212) done Nov 05 12:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_584(8171489) WEXITSTATUS 0 Nov 05 12:17:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_584(8171489) done Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_46(8175620) WEXITSTATUS 0 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_46(8175620) done Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3710(8139219) in scavenge on r908u33n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3772(8143367) in scavenge on r908u33n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3857(8144801) in scavenge on r908u35n02 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3792(8143896) in scavenge on r908u35n02 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3931(8145655) in scavenge on r908u35n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3930(8145654) in scavenge on r908u35n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3932(8145656) in scavenge on r908u35n04 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1395(8123626) in scavenge on r908u35n04 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1826(8124706) in scavenge on r908u37n01 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2219(8126107) in scavenge on r908u37n01 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2353(8126332) in scavenge on r908u37n02 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3167(8131626) in scavenge on r908u37n02 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3153(8131408) in scavenge on r908u37n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_104(8112517) in scavenge on r908u37n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1430(8123666) in scavenge on r908u37n04 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3606(8136679) in scavenge on r908u37n04 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1002(8122969) in scavenge on r908u39n02 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1344(8123526) in scavenge on r908u39n02 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3102(8130955) in scavenge on r802u06n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4041(8147094) in scavenge on r903u01n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_366(8113245) in scavenge on r903u01n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_524(8113728) in scavenge on r904u01n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4089(8148569) in scavenge on r909u07n01 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4081(8148348) in scavenge on r805u11n04 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1022(7573191) in scavenge on r805u11n04 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_621(7572790) in scavenge on r918u36n03 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_833(7573002) in scavenge on r806u09n01 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_73(7572240) in scavenge on r806u11n01 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_721(8116194) in scavenge on r807u05n02 Nov 05 12:17:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_898(7573067) in scavenge on r807u08n04 Nov 05 12:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2973(8175642) in scavenge on r816u15n09 Nov 05 12:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2974(8175643) in scavenge on r816u15n09 Nov 05 12:17:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2975(8175644) in scavenge on r816u23n09 Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_93(8170633) WEXITSTATUS 0 Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_93(8170633) done Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_16(8170556) WEXITSTATUS 0 Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_16(8170556) done Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_695(8171668) WEXITSTATUS 0 Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_695(8171668) done Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_598(8171503) WEXITSTATUS 0 Nov 05 12:17:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_598(8171503) done Nov 05 12:17:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_19(8170559) WEXITSTATUS 0 Nov 05 12:17:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_19(8170559) done Nov 05 12:17:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7786(8169194) WEXITSTATUS 1 Nov 05 12:17:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7786(8169194) done Nov 05 12:17:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169226 WEXITSTATUS 0 Nov 05 12:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169226 done Nov 05 12:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_671(8171644) WEXITSTATUS 0 Nov 05 12:17:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_671(8171644) done Nov 05 12:17:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172988 NodeList=r918u36n02 #CPUs=16 Partition=day Nov 05 12:17:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172989 NodeList=r815u35n03 #CPUs=16 Partition=day Nov 05 12:17:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172994 NodeList=r816u03n07 #CPUs=16 Partition=day Nov 05 12:17:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175635_2(8175645) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:17:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175554 NodeList=r815u09n09 #CPUs=8 Partition=scavenge Nov 05 12:17:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175555 NodeList=r816u03n12 #CPUs=8 Partition=scavenge Nov 05 12:17:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175556 NodeList=r816u03n12 #CPUs=8 Partition=scavenge Nov 05 12:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_599(8171504) WEXITSTATUS 0 Nov 05 12:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_599(8171504) done Nov 05 12:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_7(8170547) WEXITSTATUS 0 Nov 05 12:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_7(8170547) done Nov 05 12:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_591(8171496) WEXITSTATUS 0 Nov 05 12:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_591(8171496) done Nov 05 12:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175532 WEXITSTATUS 0 Nov 05 12:17:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175532 done Nov 05 12:17:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_2(8175645) WEXITSTATUS 0 Nov 05 12:17:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_2(8175645) done Nov 05 12:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_115(8170699) WEXITSTATUS 0 Nov 05 12:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_115(8170699) done Nov 05 12:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_44(8170584) WEXITSTATUS 0 Nov 05 12:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_44(8170584) done Nov 05 12:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_589(8171494) WEXITSTATUS 0 Nov 05 12:17:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_589(8171494) done Nov 05 12:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_657(8171630) WEXITSTATUS 0 Nov 05 12:18:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_657(8171630) done Nov 05 12:18:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_88(8170628) WEXITSTATUS 0 Nov 05 12:18:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_88(8170628) done Nov 05 12:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_113(8170697) WEXITSTATUS 0 Nov 05 12:18:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_113(8170697) done Nov 05 12:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_636(8171609) WEXITSTATUS 0 Nov 05 12:18:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_636(8171609) done Nov 05 12:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4032(8175640) WEXITSTATUS 0 Nov 05 12:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4032(8175640) done Nov 05 12:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7325(8166016) WEXITSTATUS 1 Nov 05 12:18:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7325(8166016) done Nov 05 12:18:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_32(8170572) WEXITSTATUS 0 Nov 05 12:18:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_32(8170572) done Nov 05 12:18:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_117(8170701) WEXITSTATUS 0 Nov 05 12:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_117(8170701) done Nov 05 12:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_86(8170626) WEXITSTATUS 0 Nov 05 12:18:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_86(8170626) done Nov 05 12:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_582(8171487) WEXITSTATUS 0 Nov 05 12:18:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_582(8171487) done Nov 05 12:18:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_45(8170585) WEXITSTATUS 0 Nov 05 12:18:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_45(8170585) done Nov 05 12:18:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_42(8170582) WEXITSTATUS 0 Nov 05 12:18:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_42(8170582) done Nov 05 12:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_629(8171602) WEXITSTATUS 0 Nov 05 12:18:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_629(8171602) done Nov 05 12:18:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_12(8170552) WEXITSTATUS 0 Nov 05 12:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_12(8170552) done Nov 05 12:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_680(8171653) WEXITSTATUS 0 Nov 05 12:18:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_680(8171653) done Nov 05 12:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_685(8171658) WEXITSTATUS 0 Nov 05 12:18:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_685(8171658) done Nov 05 12:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_638(8171611) WEXITSTATUS 0 Nov 05 12:18:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_638(8171611) done Nov 05 12:18:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_581(8171486) WEXITSTATUS 0 Nov 05 12:18:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_581(8171486) done Nov 05 12:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_40(8170580) WEXITSTATUS 0 Nov 05 12:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_40(8170580) done Nov 05 12:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8013(8171814) WEXITSTATUS 1 Nov 05 12:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8013(8171814) done Nov 05 12:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_294(8170938) WEXITSTATUS 0 Nov 05 12:18:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_294(8170938) done Nov 05 12:18:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_656(8171629) WEXITSTATUS 0 Nov 05 12:18:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_656(8171629) done Nov 05 12:18:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_1(8170541) WEXITSTATUS 0 Nov 05 12:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_1(8170541) done Nov 05 12:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_20(8170560) WEXITSTATUS 0 Nov 05 12:18:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_20(8170560) done Nov 05 12:18:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_693(8171666) WEXITSTATUS 0 Nov 05 12:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_693(8171666) done Nov 05 12:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_293(8170937) WEXITSTATUS 0 Nov 05 12:18:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_293(8170937) done Nov 05 12:18:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_17(8170557) WEXITSTATUS 0 Nov 05 12:18:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_17(8170557) done Nov 05 12:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_84(8170624) WEXITSTATUS 0 Nov 05 12:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_84(8170624) done Nov 05 12:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_19(8175448) WEXITSTATUS 0 Nov 05 12:18:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_19(8175448) done Nov 05 12:18:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_26(8170566) WEXITSTATUS 0 Nov 05 12:18:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_26(8170566) done Nov 05 12:18:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2262(8165082) WEXITSTATUS 0 Nov 05 12:18:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2262(8165082) done Nov 05 12:18:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3286(8134317) Nov 05 12:18:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2567(8152834) Nov 05 12:18:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2588(8152855) Nov 05 12:18:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_15(8170555) WEXITSTATUS 0 Nov 05 12:18:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_15(8170555) done Nov 05 12:18:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_585(8171490) WEXITSTATUS 0 Nov 05 12:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_585(8171490) done Nov 05 12:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_628(8171601) WEXITSTATUS 0 Nov 05 12:18:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_628(8171601) done Nov 05 12:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_626(8171599) WEXITSTATUS 0 Nov 05 12:18:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_626(8171599) done Nov 05 12:18:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_91(8170631) WEXITSTATUS 0 Nov 05 12:18:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_91(8170631) done Nov 05 12:18:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175646 InitPrio=34482 usec=5386 Nov 05 12:18:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_660(8171633) WEXITSTATUS 0 Nov 05 12:18:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_660(8171633) done Nov 05 12:18:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175635_3(8175647) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:18:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175557 NodeList=r815u09n11 #CPUs=8 Partition=scavenge Nov 05 12:18:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175558 NodeList=r815u23n09 #CPUs=8 Partition=scavenge Nov 05 12:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175537 WEXITSTATUS 0 Nov 05 12:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175537 done Nov 05 12:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175536 WEXITSTATUS 0 Nov 05 12:18:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175536 done Nov 05 12:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175535 WEXITSTATUS 0 Nov 05 12:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175535 done Nov 05 12:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175533 WEXITSTATUS 0 Nov 05 12:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175533 done Nov 05 12:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175534 WEXITSTATUS 0 Nov 05 12:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175534 done Nov 05 12:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_89(8170629) WEXITSTATUS 0 Nov 05 12:18:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_89(8170629) done Nov 05 12:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_745(8171800) WEXITSTATUS 0 Nov 05 12:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_745(8171800) done Nov 05 12:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_583(8171488) WEXITSTATUS 0 Nov 05 12:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_583(8171488) done Nov 05 12:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7559(8166774) WEXITSTATUS 1 Nov 05 12:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7559(8166774) done Nov 05 12:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_156(8170740) WEXITSTATUS 0 Nov 05 12:18:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_156(8170740) done Nov 05 12:18:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_3(8175647) WEXITSTATUS 0 Nov 05 12:18:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_3(8175647) done Nov 05 12:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175626 WEXITSTATUS 0 Nov 05 12:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175626 done Nov 05 12:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3930(8145654) WEXITSTATUS 1 Nov 05 12:18:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3930(8145654) done Nov 05 12:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_623(8171596) WEXITSTATUS 0 Nov 05 12:19:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_623(8171596) done Nov 05 12:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_641(8171614) WEXITSTATUS 0 Nov 05 12:19:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_641(8171614) done Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175539 WEXITSTATUS 0 Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175539 done Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_620(8171593) WEXITSTATUS 0 Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_620(8171593) done Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_295(8170939) WEXITSTATUS 0 Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_295(8170939) done Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175538 WEXITSTATUS 0 Nov 05 12:19:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175538 done Nov 05 12:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_658(8171631) WEXITSTATUS 0 Nov 05 12:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_658(8171631) done Nov 05 12:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_25(8170565) WEXITSTATUS 0 Nov 05 12:19:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_25(8170565) done Nov 05 12:19:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_624(8171597) WEXITSTATUS 0 Nov 05 12:19:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_624(8171597) done Nov 05 12:19:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_704(8171759) WEXITSTATUS 0 Nov 05 12:19:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_704(8171759) done Nov 05 12:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_2(8170542) WEXITSTATUS 0 Nov 05 12:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_2(8170542) done Nov 05 12:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_686(8171659) WEXITSTATUS 0 Nov 05 12:19:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_686(8171659) done Nov 05 12:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_129(8170713) WEXITSTATUS 0 Nov 05 12:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_129(8170713) done Nov 05 12:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_92(8170632) WEXITSTATUS 0 Nov 05 12:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_92(8170632) done Nov 05 12:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8172(8174111) WEXITSTATUS 1 Nov 05 12:19:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8172(8174111) done Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:19:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_590(8171495) WEXITSTATUS 0 Nov 05 12:19:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_590(8171495) done Nov 05 12:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_705(8171760) WEXITSTATUS 0 Nov 05 12:19:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_705(8171760) done Nov 05 12:19:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_630(8171603) WEXITSTATUS 0 Nov 05 12:19:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_630(8171603) done Nov 05 12:19:13 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 12:19:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:14 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 12:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_22(8170562) WEXITSTATUS 0 Nov 05 12:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_22(8170562) done Nov 05 12:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_0(8170540) WEXITSTATUS 0 Nov 05 12:19:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_0(8170540) done Nov 05 12:19:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7781(8169120) WEXITSTATUS 1 Nov 05 12:19:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7781(8169120) done Nov 05 12:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_96(8170636) WEXITSTATUS 0 Nov 05 12:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_96(8170636) done Nov 05 12:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_79(8170619) WEXITSTATUS 0 Nov 05 12:19:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_79(8170619) done Nov 05 12:19:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_11(8170551) WEXITSTATUS 0 Nov 05 12:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_11(8170551) done Nov 05 12:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_43(8170583) WEXITSTATUS 0 Nov 05 12:19:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_43(8170583) done Nov 05 12:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_692(8171665) WEXITSTATUS 0 Nov 05 12:19:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_692(8171665) done Nov 05 12:19:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_674(8171647) WEXITSTATUS 0 Nov 05 12:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_674(8171647) done Nov 05 12:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8062(8172500) WEXITSTATUS 1 Nov 05 12:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8062(8172500) done Nov 05 12:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2497(8165922) WEXITSTATUS 0 Nov 05 12:19:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2497(8165922) done Nov 05 12:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_697(8171670) WEXITSTATUS 0 Nov 05 12:19:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_697(8171670) done Nov 05 12:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_645(8171618) WEXITSTATUS 0 Nov 05 12:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_645(8171618) done Nov 05 12:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_642(8171615) WEXITSTATUS 0 Nov 05 12:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_642(8171615) done Nov 05 12:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_659(8171632) WEXITSTATUS 0 Nov 05 12:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_659(8171632) done Nov 05 12:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_646(8171619) WEXITSTATUS 0 Nov 05 12:19:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_646(8171619) done Nov 05 12:19:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_751(8171806) WEXITSTATUS 0 Nov 05 12:19:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_751(8171806) done Nov 05 12:19:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_296(8170940) WEXITSTATUS 0 Nov 05 12:19:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_296(8170940) done Nov 05 12:19:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_202(8170846) WEXITSTATUS 0 Nov 05 12:19:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_202(8170846) done Nov 05 12:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_299(8170943) WEXITSTATUS 0 Nov 05 12:19:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_299(8170943) done Nov 05 12:19:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:37 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8172513 Nov 05 12:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_126(8170710) WEXITSTATUS 0 Nov 05 12:19:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_126(8170710) done Nov 05 12:19:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_722(8171777) WEXITSTATUS 0 Nov 05 12:19:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_722(8171777) done Nov 05 12:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_694(8171667) WEXITSTATUS 0 Nov 05 12:19:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_694(8171667) done Nov 05 12:19:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_291(8170935) WEXITSTATUS 0 Nov 05 12:19:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_291(8170935) done Nov 05 12:19:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_640(8171613) WEXITSTATUS 0 Nov 05 12:19:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_640(8171613) done Nov 05 12:19:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_747(8171802) WEXITSTATUS 0 Nov 05 12:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_747(8171802) done Nov 05 12:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_200(8170844) WEXITSTATUS 0 Nov 05 12:19:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_200(8170844) done Nov 05 12:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_728(8171783) WEXITSTATUS 0 Nov 05 12:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_728(8171783) done Nov 05 12:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_83(8170623) WEXITSTATUS 0 Nov 05 12:19:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_83(8170623) done Nov 05 12:19:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_143(8170727) WEXITSTATUS 0 Nov 05 12:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_143(8170727) done Nov 05 12:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_652(8171625) WEXITSTATUS 0 Nov 05 12:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_652(8171625) done Nov 05 12:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_81(8170621) WEXITSTATUS 0 Nov 05 12:19:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_81(8170621) done Nov 05 12:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_163(8170747) WEXITSTATUS 0 Nov 05 12:19:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_163(8170747) done Nov 05 12:19:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_33(8170573) WEXITSTATUS 0 Nov 05 12:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_33(8170573) done Nov 05 12:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_10(8170550) WEXITSTATUS 0 Nov 05 12:19:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_10(8170550) done Nov 05 12:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_651(8171624) WEXITSTATUS 0 Nov 05 12:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_651(8171624) done Nov 05 12:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_672(8171645) WEXITSTATUS 0 Nov 05 12:19:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_672(8171645) done Nov 05 12:19:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_627(8171600) WEXITSTATUS 0 Nov 05 12:19:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_627(8171600) done Nov 05 12:19:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175635_4(8175648) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175559 NodeList=r815u15n11 #CPUs=8 Partition=scavenge Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175560 NodeList=r815u23n08 #CPUs=8 Partition=scavenge Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175561 NodeList=r816u03n10 #CPUs=8 Partition=scavenge Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175562 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175563 NodeList=r816u03n11 #CPUs=8 Partition=scavenge Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175564 NodeList=r816u23n07 #CPUs=8 Partition=scavenge Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175565 NodeList=r816u23n09 #CPUs=8 Partition=scavenge Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175540 WEXITSTATUS 0 Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175540 done Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_21(8170561) WEXITSTATUS 0 Nov 05 12:19:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_21(8170561) done Nov 05 12:19:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175541 WEXITSTATUS 0 Nov 05 12:19:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175541 done Nov 05 12:19:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_146(8170730) WEXITSTATUS 0 Nov 05 12:19:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_146(8170730) done Nov 05 12:19:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_4(8175648) WEXITSTATUS 0 Nov 05 12:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_4(8175648) done Nov 05 12:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_675(8171648) WEXITSTATUS 0 Nov 05 12:20:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_675(8171648) done Nov 05 12:20:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_99(8170639) WEXITSTATUS 0 Nov 05 12:20:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_99(8170639) done Nov 05 12:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_292(8170936) WEXITSTATUS 0 Nov 05 12:20:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_292(8170936) done Nov 05 12:20:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175542 WEXITSTATUS 0 Nov 05 12:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175542 done Nov 05 12:20:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175649 InitPrio=10612 usec=4369 Nov 05 12:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175543 WEXITSTATUS 0 Nov 05 12:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175543 done Nov 05 12:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_77(8170617) WEXITSTATUS 0 Nov 05 12:20:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_77(8170617) done Nov 05 12:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175544 WEXITSTATUS 0 Nov 05 12:20:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175544 done Nov 05 12:20:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_154(8170738) WEXITSTATUS 0 Nov 05 12:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_154(8170738) done Nov 05 12:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_3(8170543) WEXITSTATUS 0 Nov 05 12:20:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_3(8170543) done Nov 05 12:20:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_650(8171623) WEXITSTATUS 0 Nov 05 12:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_650(8171623) done Nov 05 12:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_649(8171622) WEXITSTATUS 0 Nov 05 12:20:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_649(8171622) done Nov 05 12:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_4(8170544) WEXITSTATUS 0 Nov 05 12:20:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_4(8170544) done Nov 05 12:20:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_698(8171671) WEXITSTATUS 0 Nov 05 12:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_698(8171671) done Nov 05 12:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_731(8171786) WEXITSTATUS 0 Nov 05 12:20:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_731(8171786) done Nov 05 12:20:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175650 InitPrio=10612 usec=4405 Nov 05 12:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_258(8170902) WEXITSTATUS 0 Nov 05 12:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_258(8170902) done Nov 05 12:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_667(8171640) WEXITSTATUS 0 Nov 05 12:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_667(8171640) done Nov 05 12:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_313(8171032) WEXITSTATUS 0 Nov 05 12:20:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_313(8171032) done Nov 05 12:20:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_648(8171621) WEXITSTATUS 0 Nov 05 12:20:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_648(8171621) done Nov 05 12:20:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175631 WEXITSTATUS 0 Nov 05 12:20:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175631 done Nov 05 12:20:19 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1700 Nov 05 12:20:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_23(8170563) WEXITSTATUS 0 Nov 05 12:20:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_23(8170563) done Nov 05 12:20:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_739(8171794) WEXITSTATUS 0 Nov 05 12:20:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_739(8171794) done Nov 05 12:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_752(8171807) WEXITSTATUS 0 Nov 05 12:20:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_752(8171807) done Nov 05 12:20:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172995 in day on r914u01n02 Nov 05 12:20:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8172998 in day on r816u09n03 Nov 05 12:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_35(8170575) WEXITSTATUS 0 Nov 05 12:20:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_35(8170575) done Nov 05 12:20:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175635_5(8175651) in bigmem on r807u39n01 Nov 05 12:20:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4034(8175652) in gpu on r903u34n01 Nov 05 12:20:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4035(8175653) in gpu on r903u40n01 Nov 05 12:20:25 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 12:20:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175649 in day on r906u41n04 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175650 in day on r904u07n03 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175566 in scavenge on r816u03n09 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175567 in scavenge on r816u15n09 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175568 in scavenge on r816u29n03 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175569 in scavenge on r816u29n03 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175570 in scavenge on r816u29n03 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175571 in scavenge on r815u23n02 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175572 in scavenge on r815u29n02 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175573 in scavenge on r815u35n03 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175574 in scavenge on r816u09n05 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175575 in scavenge on r816u09n06 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175576 in scavenge on r816u09n07 Nov 05 12:20:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175577 in scavenge on r816u09n07 Nov 05 12:20:28 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144163 in scavenge on r914u13n02,r916u01n04 Nov 05 12:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_5(8175651) WEXITSTATUS 0 Nov 05 12:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_5(8175651) done Nov 05 12:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_127(8170711) WEXITSTATUS 0 Nov 05 12:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_127(8170711) done Nov 05 12:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_668(8171641) WEXITSTATUS 0 Nov 05 12:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_668(8171641) done Nov 05 12:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_147(8170731) WEXITSTATUS 0 Nov 05 12:20:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_147(8170731) done Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_725(8171780) WEXITSTATUS 0 Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_725(8171780) done Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_753(8171808) WEXITSTATUS 0 Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_753(8171808) done Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_733(8171788) WEXITSTATUS 0 Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_733(8171788) done Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_14(8170554) WEXITSTATUS 0 Nov 05 12:20:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_14(8170554) done Nov 05 12:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175545 WEXITSTATUS 0 Nov 05 12:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175545 done Nov 05 12:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175551 WEXITSTATUS 0 Nov 05 12:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175551 done Nov 05 12:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175547 WEXITSTATUS 0 Nov 05 12:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175547 done Nov 05 12:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175546 WEXITSTATUS 0 Nov 05 12:20:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175546 done Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175549 WEXITSTATUS 0 Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175549 done Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_676(8171649) WEXITSTATUS 0 Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_676(8171649) done Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175552 WEXITSTATUS 0 Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175552 done Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_666(8171639) WEXITSTATUS 0 Nov 05 12:20:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_666(8171639) done Nov 05 12:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_94(8170634) WEXITSTATUS 0 Nov 05 12:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_94(8170634) done Nov 05 12:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_95(8170635) WEXITSTATUS 0 Nov 05 12:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_95(8170635) done Nov 05 12:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175550 WEXITSTATUS 0 Nov 05 12:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175550 done Nov 05 12:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175553 WEXITSTATUS 0 Nov 05 12:20:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175553 done Nov 05 12:20:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_635(8171608) WEXITSTATUS 0 Nov 05 12:20:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_635(8171608) done Nov 05 12:20:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_625(8171598) WEXITSTATUS 0 Nov 05 12:20:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_625(8171598) done Nov 05 12:20:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175548 WEXITSTATUS 0 Nov 05 12:20:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175548 done Nov 05 12:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_201(8170845) WEXITSTATUS 0 Nov 05 12:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_201(8170845) done Nov 05 12:20:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7311(8165995) WEXITSTATUS 1 Nov 05 12:20:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7311(8165995) done Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3618(8136983) in scavenge on r907u05n01 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3521(8135652) in scavenge on r908u35n03 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_757(8114263) in scavenge on r904u01n02 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1098(8123199) in scavenge on r802u11n02 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3029(8129034) in scavenge on r802u11n04 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3829(8144443) in scavenge on r805u27n03 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4032(8146702) in scavenge on r807u05n02 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1222(8123362) in scavenge on r904u04n03 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1419(8123652) in scavenge on r914u05n03 Nov 05 12:20:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3868(8144933) in scavenge on r919u01n04 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2976(8175654) in scavenge on r815u15n11 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2977(8175655) in scavenge on r815u15n11 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2978(8175656) in scavenge on r815u23n08 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2979(8175657) in scavenge on r815u23n08 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2980(8175658) in scavenge on r815u23n09 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2981(8175659) in scavenge on r815u23n09 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2982(8175660) in scavenge on r816u03n10 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2983(8175661) in scavenge on r816u03n10 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2984(8175662) in scavenge on r816u03n12 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2985(8175663) in scavenge on r816u03n12 Nov 05 12:20:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2986(8175664) in scavenge on r816u03n12 Nov 05 12:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_353(8171072) WEXITSTATUS 0 Nov 05 12:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_353(8171072) done Nov 05 12:20:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_351(8171070) WEXITSTATUS 0 Nov 05 12:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_351(8171070) done Nov 05 12:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3686(8138482) WEXITSTATUS 1 Nov 05 12:20:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3686(8138482) done Nov 05 12:20:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_723(8171778) WEXITSTATUS 0 Nov 05 12:20:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_723(8171778) done Nov 05 12:20:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_173(8170757) WEXITSTATUS 0 Nov 05 12:20:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_173(8170757) done Nov 05 12:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2484(8165851) WEXITSTATUS 0 Nov 05 12:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2484(8165851) done Nov 05 12:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3736(8141693) WEXITSTATUS 1 Nov 05 12:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3736(8141693) done Nov 05 12:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_746(8171801) WEXITSTATUS 0 Nov 05 12:20:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_746(8171801) done Nov 05 12:20:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_345(8171064) WEXITSTATUS 0 Nov 05 12:20:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_345(8171064) done Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175554 WEXITSTATUS 0 Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175554 done Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175635_6(8175665) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175578 NodeList=r816u09n09 #CPUs=8 Partition=scavenge Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175579 NodeList=r816u23n11 #CPUs=8 Partition=scavenge Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175580 NodeList=r815u09n09 #CPUs=8 Partition=scavenge Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175581 NodeList=r815u09n10 #CPUs=8 Partition=scavenge Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175582 NodeList=r816u23n06 #CPUs=8 Partition=scavenge Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175583 NodeList=r816u29n02 #CPUs=8 Partition=scavenge Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175555 WEXITSTATUS 0 Nov 05 12:20:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175555 done Nov 05 12:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_6(8175665) WEXITSTATUS 0 Nov 05 12:20:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_6(8175665) done Nov 05 12:20:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175556 WEXITSTATUS 0 Nov 05 12:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175556 done Nov 05 12:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_104(8170688) WEXITSTATUS 0 Nov 05 12:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_104(8170688) done Nov 05 12:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_689(8171662) WEXITSTATUS 0 Nov 05 12:20:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_689(8171662) done Nov 05 12:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_669(8171642) WEXITSTATUS 0 Nov 05 12:21:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_669(8171642) done Nov 05 12:21:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_68(8170608) WEXITSTATUS 0 Nov 05 12:21:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_68(8170608) done Nov 05 12:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_190(8170774) WEXITSTATUS 0 Nov 05 12:21:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_190(8170774) done Nov 05 12:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_741(8171796) WEXITSTATUS 0 Nov 05 12:21:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_741(8171796) done Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_124(8170708) WEXITSTATUS 0 Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_124(8170708) done Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_732(8171787) WEXITSTATUS 0 Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_732(8171787) done Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_130(8170714) WEXITSTATUS 0 Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_130(8170714) done Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_133(8170717) WEXITSTATUS 0 Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_133(8170717) done Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_740(8171795) WEXITSTATUS 0 Nov 05 12:21:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_740(8171795) done Nov 05 12:21:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_310(8171029) WEXITSTATUS 0 Nov 05 12:21:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_310(8171029) done Nov 05 12:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8059(8172497) WEXITSTATUS 1 Nov 05 12:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8059(8172497) done Nov 05 12:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_145(8170729) WEXITSTATUS 0 Nov 05 12:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_145(8170729) done Nov 05 12:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_82(8170622) WEXITSTATUS 0 Nov 05 12:21:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_82(8170622) done Nov 05 12:21:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8125756 Nov 05 12:21:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_719(8171774) WEXITSTATUS 0 Nov 05 12:21:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_719(8171774) done Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_85(8170625) WEXITSTATUS 0 Nov 05 12:21:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_85(8170625) done Nov 05 12:21:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175666 InitPrio=39776 usec=4248 Nov 05 12:21:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175666 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 12:21:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_161(8170745) WEXITSTATUS 0 Nov 05 12:21:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_161(8170745) done Nov 05 12:21:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_159(8170743) WEXITSTATUS 0 Nov 05 12:21:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_159(8170743) done Nov 05 12:21:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_122(8170706) WEXITSTATUS 0 Nov 05 12:21:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_122(8170706) done Nov 05 12:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_179(8170763) WEXITSTATUS 0 Nov 05 12:21:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_179(8170763) done Nov 05 12:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_647(8171620) WEXITSTATUS 0 Nov 05 12:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_647(8171620) done Nov 05 12:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_142(8170726) WEXITSTATUS 0 Nov 05 12:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_142(8170726) done Nov 05 12:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_707(8171762) WEXITSTATUS 0 Nov 05 12:21:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_707(8171762) done Nov 05 12:21:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_634(8171607) WEXITSTATUS 0 Nov 05 12:21:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_634(8171607) done Nov 05 12:21:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_49(8174123) WEXITSTATUS 0 Nov 05 12:21:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_49(8174123) done Nov 05 12:21:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_311(8171030) WEXITSTATUS 0 Nov 05 12:21:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_311(8171030) done Nov 05 12:21:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_177(8170761) WEXITSTATUS 0 Nov 05 12:21:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_177(8170761) done Nov 05 12:21:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_180(8170764) WEXITSTATUS 0 Nov 05 12:21:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_180(8170764) done Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_749(8171804) WEXITSTATUS 0 Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_749(8171804) done Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_123(8170707) WEXITSTATUS 0 Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_123(8170707) done Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_176(8170760) WEXITSTATUS 0 Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_176(8170760) done Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_663(8171636) WEXITSTATUS 0 Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_663(8171636) done Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_729(8171784) WEXITSTATUS 0 Nov 05 12:21:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_729(8171784) done Nov 05 12:21:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_735(8171790) WEXITSTATUS 0 Nov 05 12:21:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_735(8171790) done Nov 05 12:21:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_170(8170754) WEXITSTATUS 0 Nov 05 12:21:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_170(8170754) done Nov 05 12:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_706(8171761) WEXITSTATUS 0 Nov 05 12:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_706(8171761) done Nov 05 12:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_73(8170613) WEXITSTATUS 0 Nov 05 12:21:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_73(8170613) done Nov 05 12:21:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_682(8171655) WEXITSTATUS 0 Nov 05 12:21:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_682(8171655) done Nov 05 12:21:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2252(8165056) WEXITSTATUS 0 Nov 05 12:21:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2252(8165056) done Nov 05 12:21:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_134(8170718) WEXITSTATUS 0 Nov 05 12:21:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_134(8170718) done Nov 05 12:21:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_338(8171057) WEXITSTATUS 0 Nov 05 12:21:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_338(8171057) done Nov 05 12:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_76(8170616) WEXITSTATUS 0 Nov 05 12:21:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_76(8170616) done Nov 05 12:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_691(8171664) WEXITSTATUS 0 Nov 05 12:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_691(8171664) done Nov 05 12:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_144(8170728) WEXITSTATUS 0 Nov 05 12:21:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_144(8170728) done Nov 05 12:21:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_738(8171793) WEXITSTATUS 0 Nov 05 12:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_738(8171793) done Nov 05 12:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_342(8171061) WEXITSTATUS 0 Nov 05 12:21:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_342(8171061) done Nov 05 12:21:39 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_721(8171776) WEXITSTATUS 0 Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_721(8171776) done Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_343(8171062) WEXITSTATUS 0 Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_343(8171062) done Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_344(8171063) WEXITSTATUS 0 Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_344(8171063) done Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_189(8170773) WEXITSTATUS 0 Nov 05 12:21:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_189(8170773) done Nov 05 12:21:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_167(8170751) WEXITSTATUS 0 Nov 05 12:21:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_167(8170751) done Nov 05 12:21:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_349(8171068) WEXITSTATUS 0 Nov 05 12:21:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_349(8171068) done Nov 05 12:21:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_100(8170684) WEXITSTATUS 0 Nov 05 12:21:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_100(8170684) done Nov 05 12:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_140(8170724) WEXITSTATUS 0 Nov 05 12:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_140(8170724) done Nov 05 12:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_181(8170765) WEXITSTATUS 0 Nov 05 12:21:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_181(8170765) done Nov 05 12:21:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_330(8171049) WEXITSTATUS 0 Nov 05 12:21:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_330(8171049) done Nov 05 12:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_734(8171789) WEXITSTATUS 0 Nov 05 12:21:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_734(8171789) done Nov 05 12:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_720(8171775) WEXITSTATUS 0 Nov 05 12:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_720(8171775) done Nov 05 12:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_78(8170618) WEXITSTATUS 0 Nov 05 12:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_78(8170618) done Nov 05 12:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_730(8171785) WEXITSTATUS 0 Nov 05 12:21:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_730(8171785) done Nov 05 12:21:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_70(8170610) WEXITSTATUS 0 Nov 05 12:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_70(8170610) done Nov 05 12:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_637(8171610) WEXITSTATUS 0 Nov 05 12:21:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_637(8171610) done Nov 05 12:21:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_175(8170759) WEXITSTATUS 0 Nov 05 12:21:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_175(8170759) done Nov 05 12:21:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_125(8170709) WEXITSTATUS 0 Nov 05 12:21:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_125(8170709) done Nov 05 12:21:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_727(8171782) WEXITSTATUS 0 Nov 05 12:21:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_727(8171782) done Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175646 NodeList=r815u03n01 #CPUs=30 Partition=day Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172996 NodeList=r816u09n01 #CPUs=16 Partition=day Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172997 NodeList=r816u09n01 #CPUs=16 Partition=day Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8172999 NodeList=r815u03n01 #CPUs=16 Partition=day Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173000 NodeList=r815u15n04 #CPUs=16 Partition=day Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175635_7(8175667) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175584 NodeList=r816u03n12 #CPUs=8 Partition=scavenge Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175585 NodeList=r816u03n12 #CPUs=8 Partition=scavenge Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175557 WEXITSTATUS 0 Nov 05 12:21:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175557 done Nov 05 12:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175558 WEXITSTATUS 0 Nov 05 12:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175558 done Nov 05 12:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_335(8171054) WEXITSTATUS 0 Nov 05 12:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_335(8171054) done Nov 05 12:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_326(8171045) WEXITSTATUS 0 Nov 05 12:21:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_326(8171045) done Nov 05 12:21:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_7(8175667) WEXITSTATUS 0 Nov 05 12:21:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_7(8175667) done Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175668 InitPrio=13477 usec=1923 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175669 InitPrio=13477 usec=1862 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_1(unfulfilled) for JobId=8175669_1(8175670) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_1 uid=23340 usec=207 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_2(unfulfilled) for JobId=8175669_2(8175671) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_2 uid=23340 usec=264 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_3(unfulfilled) for JobId=8175669_3(8175672) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_3 uid=23340 usec=183 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_4(unfulfilled) for JobId=8175669_4(8175673) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_4 uid=23340 usec=192 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_5(unfulfilled) for JobId=8175669_5(8175674) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_5 uid=23340 usec=159 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_6(unfulfilled) for JobId=8175669_6(8175675) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_6 uid=23340 usec=261 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_7(unfulfilled) for JobId=8175669_7(8175676) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_7 uid=23340 usec=165 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_8(unfulfilled) for JobId=8175669_8(8175677) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_8 uid=23340 usec=339 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_9(unfulfilled) for JobId=8175669_9(8175678) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_9 uid=23340 usec=155 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_10(unfulfilled) for JobId=8175669_10(8175679) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_10 uid=23340 usec=137 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_11(unfulfilled) for JobId=8175669_11(8175680) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_11 uid=23340 usec=122 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_12(unfulfilled) for JobId=8175669_12(8175681) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_12 uid=23340 usec=155 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_13(unfulfilled) for JobId=8175669_13(8175682) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_13 uid=23340 usec=142 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_14(unfulfilled) for JobId=8175669_14(8175683) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_14 uid=23340 usec=136 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_15(unfulfilled) for JobId=8175669_15(8175684) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_15 uid=23340 usec=161 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_16(unfulfilled) for JobId=8175669_16(8175685) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_16 uid=23340 usec=148 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_17(unfulfilled) for JobId=8175669_17(8175686) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_17 uid=23340 usec=168 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4031(8175633) WEXITSTATUS 0 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4031(8175633) done Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_18(unfulfilled) for JobId=8175669_18(8175687) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_18 uid=23340 usec=214 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_19(unfulfilled) for JobId=8175669_19(8175688) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_19 uid=23340 usec=130 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_20(unfulfilled) for JobId=8175669_20(8175689) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_20 uid=23340 usec=142 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_21(unfulfilled) for JobId=8175669_21(8175690) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_21 uid=23340 usec=120 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_22(unfulfilled) for JobId=8175669_22(8175691) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_22 uid=23340 usec=151 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_23(unfulfilled) for JobId=8175669_23(8175692) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_23 uid=23340 usec=215 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_24(unfulfilled) for JobId=8175669_24(8175693) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_24 uid=23340 usec=121 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_25(unfulfilled) for JobId=8175669_25(8175694) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_25 uid=23340 usec=123 Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_26(unfulfilled) for JobId=8175669_26(8175695) Nov 05 12:21:59 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_26 uid=23340 usec=118 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_27(unfulfilled) for JobId=8175669_27(8175696) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_27 uid=23340 usec=10978 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_28(unfulfilled) for JobId=8175669_28(8175697) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_28 uid=23340 usec=152 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_29(unfulfilled) for JobId=8175669_29(8175698) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_29 uid=23340 usec=141 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_30(unfulfilled) for JobId=8175669_30(8175699) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_30 uid=23340 usec=123 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_31(unfulfilled) for JobId=8175669_31(8175700) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_31 uid=23340 usec=127 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_32(unfulfilled) for JobId=8175669_32(8175701) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_32 uid=23340 usec=118 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_33(unfulfilled) for JobId=8175669_33(8175702) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_33 uid=23340 usec=118 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_34(unfulfilled) for JobId=8175669_34(8175703) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_34 uid=23340 usec=146 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_35(unfulfilled) for JobId=8175669_35(8175704) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_35 uid=23340 usec=110 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_36(unfulfilled) for JobId=8175669_36(8175705) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_36 uid=23340 usec=112 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_37(unfulfilled) for JobId=8175669_37(8175706) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_37 uid=23340 usec=40973 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_38(unfulfilled) for JobId=8175669_38(8175707) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_38 uid=23340 usec=176 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_39(unfulfilled) for JobId=8175669_39(8175708) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_39 uid=23340 usec=131 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_40(unfulfilled) for JobId=8175669_40(8175709) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_40 uid=23340 usec=118 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_41(unfulfilled) for JobId=8175669_41(8175710) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_41 uid=23340 usec=130 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_42(unfulfilled) for JobId=8175669_42(8175711) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_42 uid=23340 usec=121 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_43(unfulfilled) for JobId=8175669_43(8175712) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_43 uid=23340 usec=126 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_44(unfulfilled) for JobId=8175669_44(8175713) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_44 uid=23340 usec=119 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_45(unfulfilled) for JobId=8175669_45(8175714) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_45 uid=23340 usec=121 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_46(unfulfilled) for JobId=8175669_46(8175715) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_46 uid=23340 usec=116 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_47(unfulfilled) for JobId=8175669_47(8175716) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_47 uid=23340 usec=122 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_48(unfulfilled) for JobId=8175669_48(8175717) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_48 uid=23340 usec=122 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_49(unfulfilled) for JobId=8175669_49(8175718) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_49 uid=23340 usec=120 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_50(unfulfilled) for JobId=8175669_50(8175719) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_50 uid=23340 usec=117 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_51(unfulfilled) for JobId=8175669_51(8175720) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_51 uid=23340 usec=114 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_52(unfulfilled) for JobId=8175669_52(8175721) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_52 uid=23340 usec=111 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_53(unfulfilled) for JobId=8175669_53(8175722) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_53 uid=23340 usec=115 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_54(unfulfilled) for JobId=8175669_54(8175723) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_54 uid=23340 usec=116 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_55(unfulfilled) for JobId=8175669_55(8175724) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_55 uid=23340 usec=125 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_56(unfulfilled) for JobId=8175669_56(8175725) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_56 uid=23340 usec=120 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_57(unfulfilled) for JobId=8175669_57(8175726) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_57 uid=23340 usec=119 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_58(unfulfilled) for JobId=8175669_58(8175727) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_58 uid=23340 usec=146 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_59(unfulfilled) for JobId=8175669_59(8175728) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_59 uid=23340 usec=165 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_60(unfulfilled) for JobId=8175669_60(8175729) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_60 uid=23340 usec=176 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_61(unfulfilled) for JobId=8175669_61(8175730) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_61 uid=23340 usec=121 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_62(unfulfilled) for JobId=8175669_62(8175731) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_62 uid=23340 usec=117 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_63(unfulfilled) for JobId=8175669_63(8175732) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_63 uid=23340 usec=130 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_64(unfulfilled) for JobId=8175669_64(8175733) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_64 uid=23340 usec=143 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_65(unfulfilled) for JobId=8175669_65(8175734) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_65 uid=23340 usec=117 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_66(unfulfilled) for JobId=8175669_66(8175735) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_66 uid=23340 usec=112 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_67(unfulfilled) for JobId=8175669_67(8175736) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_67 uid=23340 usec=118 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_68(unfulfilled) for JobId=8175669_68(8175737) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_68 uid=23340 usec=116 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_69(unfulfilled) for JobId=8175669_69(8175738) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_69 uid=23340 usec=114 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_70(unfulfilled) for JobId=8175669_70(8175739) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_70 uid=23340 usec=123 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_71(unfulfilled) for JobId=8175669_71(8175740) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_71 uid=23340 usec=118 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_72(unfulfilled) for JobId=8175669_72(8175741) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_72 uid=23340 usec=115 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_73(unfulfilled) for JobId=8175669_73(8175742) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_73 uid=23340 usec=115 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_74(unfulfilled) for JobId=8175669_74(8175743) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_74 uid=23340 usec=112 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_75(unfulfilled) for JobId=8175669_75(8175744) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_75 uid=23340 usec=984 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_76(unfulfilled) for JobId=8175669_76(8175745) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_76 uid=23340 usec=210 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_77(unfulfilled) for JobId=8175669_77(8175746) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_77 uid=23340 usec=119 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_78(unfulfilled) for JobId=8175669_78(8175747) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_78 uid=23340 usec=116 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_79(unfulfilled) for JobId=8175669_79(8175748) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_79 uid=23340 usec=115 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_80(unfulfilled) for JobId=8175669_80(8175749) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_80 uid=23340 usec=116 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_81(unfulfilled) for JobId=8175669_81(8175750) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_81 uid=23340 usec=116 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_82(unfulfilled) for JobId=8175669_82(8175751) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_82 uid=23340 usec=119 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_83(unfulfilled) for JobId=8175669_83(8175752) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_83 uid=23340 usec=133 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_84(unfulfilled) for JobId=8175669_84(8175753) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_84 uid=23340 usec=119 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_85(unfulfilled) for JobId=8175669_85(8175754) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_85 uid=23340 usec=301 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_86(unfulfilled) for JobId=8175669_86(8175755) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_86 uid=23340 usec=116 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_87(unfulfilled) for JobId=8175669_87(8175756) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_87 uid=23340 usec=114 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_88(unfulfilled) for JobId=8175669_88(8175757) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_88 uid=23340 usec=116 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_89(unfulfilled) for JobId=8175669_89(8175758) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_89 uid=23340 usec=121 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_90(unfulfilled) for JobId=8175669_90(8175759) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_90 uid=23340 usec=1047 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_91(unfulfilled) for JobId=8175669_91(8175760) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_91 uid=23340 usec=35426 Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_92(unfulfilled) for JobId=8175669_92(8175761) Nov 05 12:22:00 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_92 uid=23340 usec=4009 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_93(unfulfilled) for JobId=8175669_93(8175762) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_93 uid=23340 usec=35793 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_94(unfulfilled) for JobId=8175669_94(8175763) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_94 uid=23340 usec=173 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_95(unfulfilled) for JobId=8175669_95(8175764) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_95 uid=23340 usec=124 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175668_96(unfulfilled) for JobId=8175669_96(8175669) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175669_96 uid=23340 usec=109 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175765 InitPrio=13477 usec=1691 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_1(unfulfilled) for JobId=8175765_1(8175766) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_1 uid=23340 usec=125 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_2(unfulfilled) for JobId=8175765_2(8175767) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_2 uid=23340 usec=140 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_3(unfulfilled) for JobId=8175765_3(8175768) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_3 uid=23340 usec=129 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_4(unfulfilled) for JobId=8175765_4(8175769) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_4 uid=23340 usec=124 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_5(unfulfilled) for JobId=8175765_5(8175770) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_5 uid=23340 usec=119 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_6(unfulfilled) for JobId=8175765_6(8175771) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_6 uid=23340 usec=147 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_7(unfulfilled) for JobId=8175765_7(8175772) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_7 uid=23340 usec=122 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_8(unfulfilled) for JobId=8175765_8(8175773) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_8 uid=23340 usec=117 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_9(unfulfilled) for JobId=8175765_9(8175774) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_9 uid=23340 usec=115 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_10(unfulfilled) for JobId=8175765_10(8175775) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_10 uid=23340 usec=133 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_11(unfulfilled) for JobId=8175765_11(8175776) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_11 uid=23340 usec=114 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_12(unfulfilled) for JobId=8175765_12(8175777) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_12 uid=23340 usec=111 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_13(unfulfilled) for JobId=8175765_13(8175778) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_13 uid=23340 usec=124 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_14(unfulfilled) for JobId=8175765_14(8175779) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_14 uid=23340 usec=111 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_15(unfulfilled) for JobId=8175765_15(8175780) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_15 uid=23340 usec=116 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_355(8171074) WEXITSTATUS 0 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_355(8171074) done Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_16(unfulfilled) for JobId=8175765_16(8175781) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_16 uid=23340 usec=141 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_17(unfulfilled) for JobId=8175765_17(8175782) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_17 uid=23340 usec=331 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_18(unfulfilled) for JobId=8175765_18(8175783) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_18 uid=23340 usec=130 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_19(unfulfilled) for JobId=8175765_19(8175784) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_19 uid=23340 usec=124 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_20(unfulfilled) for JobId=8175765_20(8175785) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_20 uid=23340 usec=122 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_21(unfulfilled) for JobId=8175765_21(8175786) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_21 uid=23340 usec=126 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_22(unfulfilled) for JobId=8175765_22(8175787) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_22 uid=23340 usec=123 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_23(unfulfilled) for JobId=8175765_23(8175788) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_23 uid=23340 usec=171 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_24(unfulfilled) for JobId=8175765_24(8175789) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_24 uid=23340 usec=312 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_25(unfulfilled) for JobId=8175765_25(8175790) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_25 uid=23340 usec=147 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_26(unfulfilled) for JobId=8175765_26(8175791) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_26 uid=23340 usec=126 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_27(unfulfilled) for JobId=8175765_27(8175792) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_27 uid=23340 usec=200 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_28(unfulfilled) for JobId=8175765_28(8175793) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_28 uid=23340 usec=112 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_29(unfulfilled) for JobId=8175765_29(8175794) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_29 uid=23340 usec=112 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_30(unfulfilled) for JobId=8175765_30(8175795) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_30 uid=23340 usec=118 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_31(unfulfilled) for JobId=8175765_31(8175796) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_31 uid=23340 usec=126 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_32(unfulfilled) for JobId=8175765_32(8175797) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_32 uid=23340 usec=117 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_33(unfulfilled) for JobId=8175765_33(8175798) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_33 uid=23340 usec=163 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_34(unfulfilled) for JobId=8175765_34(8175799) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_34 uid=23340 usec=120 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_35(unfulfilled) for JobId=8175765_35(8175800) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_35 uid=23340 usec=199 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_36(unfulfilled) for JobId=8175765_36(8175801) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_36 uid=23340 usec=119 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_37(unfulfilled) for JobId=8175765_37(8175802) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_37 uid=23340 usec=118 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_38(unfulfilled) for JobId=8175765_38(8175803) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_38 uid=23340 usec=132 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_39(unfulfilled) for JobId=8175765_39(8175804) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_39 uid=23340 usec=124 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_40(unfulfilled) for JobId=8175765_40(8175805) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_40 uid=23340 usec=119 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_41(unfulfilled) for JobId=8175765_41(8175806) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_41 uid=23340 usec=119 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_42(unfulfilled) for JobId=8175765_42(8175807) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_42 uid=23340 usec=115 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_43(unfulfilled) for JobId=8175765_43(8175808) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_43 uid=23340 usec=164 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_44(unfulfilled) for JobId=8175765_44(8175809) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_44 uid=23340 usec=34740 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_45(unfulfilled) for JobId=8175765_45(8175810) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_45 uid=23340 usec=147 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_46(unfulfilled) for JobId=8175765_46(8175811) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_46 uid=23340 usec=126 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_47(unfulfilled) for JobId=8175765_47(8175812) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_47 uid=23340 usec=123 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_48(unfulfilled) for JobId=8175765_48(8175813) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_48 uid=23340 usec=122 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_49(unfulfilled) for JobId=8175765_49(8175814) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_49 uid=23340 usec=125 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_50(unfulfilled) for JobId=8175765_50(8175815) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_50 uid=23340 usec=118 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_51(unfulfilled) for JobId=8175765_51(8175816) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_51 uid=23340 usec=121 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_52(unfulfilled) for JobId=8175765_52(8175817) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_52 uid=23340 usec=121 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_53(unfulfilled) for JobId=8175765_53(8175818) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_53 uid=23340 usec=119 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_54(unfulfilled) for JobId=8175765_54(8175819) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_54 uid=23340 usec=122 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_55(unfulfilled) for JobId=8175765_55(8175820) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_55 uid=23340 usec=120 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_56(unfulfilled) for JobId=8175765_56(8175821) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_56 uid=23340 usec=117 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_57(unfulfilled) for JobId=8175765_57(8175822) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_57 uid=23340 usec=119 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_58(unfulfilled) for JobId=8175765_58(8175823) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_58 uid=23340 usec=131 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_59(unfulfilled) for JobId=8175765_59(8175824) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_59 uid=23340 usec=127 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_60(unfulfilled) for JobId=8175765_60(8175825) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_60 uid=23340 usec=122 Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_61(unfulfilled) for JobId=8175765_61(8175826) Nov 05 12:22:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_61 uid=23340 usec=119 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_62(unfulfilled) for JobId=8175765_62(8175827) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_62 uid=23340 usec=106115 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_63(unfulfilled) for JobId=8175765_63(8175828) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_63 uid=23340 usec=228 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_64(unfulfilled) for JobId=8175765_64(8175829) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_64 uid=23340 usec=121 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_65(unfulfilled) for JobId=8175765_65(8175830) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_65 uid=23340 usec=121 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_66(unfulfilled) for JobId=8175765_66(8175831) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_66 uid=23340 usec=119 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_67(unfulfilled) for JobId=8175765_67(8175832) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_67 uid=23340 usec=115 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_68(unfulfilled) for JobId=8175765_68(8175833) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_68 uid=23340 usec=113 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_69(unfulfilled) for JobId=8175765_69(8175834) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_69 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_70(unfulfilled) for JobId=8175765_70(8175835) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_70 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_71(unfulfilled) for JobId=8175765_71(8175836) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_71 uid=23340 usec=112 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_72(unfulfilled) for JobId=8175765_72(8175837) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_72 uid=23340 usec=199 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_73(unfulfilled) for JobId=8175765_73(8175838) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_73 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_74(unfulfilled) for JobId=8175765_74(8175839) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_74 uid=23340 usec=109 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_75(unfulfilled) for JobId=8175765_75(8175840) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_75 uid=23340 usec=107 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_76(unfulfilled) for JobId=8175765_76(8175841) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_76 uid=23340 usec=291 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_77(unfulfilled) for JobId=8175765_77(8175842) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_77 uid=23340 usec=108 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_78(unfulfilled) for JobId=8175765_78(8175843) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_78 uid=23340 usec=114 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_79(unfulfilled) for JobId=8175765_79(8175844) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_79 uid=23340 usec=114 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_80(unfulfilled) for JobId=8175765_80(8175845) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_80 uid=23340 usec=110 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_81(unfulfilled) for JobId=8175765_81(8175846) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_81 uid=23340 usec=304 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_82(unfulfilled) for JobId=8175765_82(8175847) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_82 uid=23340 usec=108 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_83(unfulfilled) for JobId=8175765_83(8175848) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_83 uid=23340 usec=108 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_84(unfulfilled) for JobId=8175765_84(8175849) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_84 uid=23340 usec=191 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_85(unfulfilled) for JobId=8175765_85(8175850) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_85 uid=23340 usec=106 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_86(unfulfilled) for JobId=8175765_86(8175851) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_86 uid=23340 usec=109 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_87(unfulfilled) for JobId=8175765_87(8175852) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_87 uid=23340 usec=157 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_88(unfulfilled) for JobId=8175765_88(8175853) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_88 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_89(unfulfilled) for JobId=8175765_89(8175854) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_89 uid=23340 usec=37231 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_90(unfulfilled) for JobId=8175765_90(8175855) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_90 uid=23340 usec=135 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_91(unfulfilled) for JobId=8175765_91(8175856) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_91 uid=23340 usec=128 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_92(unfulfilled) for JobId=8175765_92(8175857) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_92 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_93(unfulfilled) for JobId=8175765_93(8175858) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_93 uid=23340 usec=112 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_94(unfulfilled) for JobId=8175765_94(8175859) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_94 uid=23340 usec=112 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_95(unfulfilled) for JobId=8175765_95(8175860) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_95 uid=23340 usec=113 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175669_96(unfulfilled) for JobId=8175765_96(8175765) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175765_96 uid=23340 usec=105 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175861 InitPrio=13477 usec=2243 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_1(unfulfilled) for JobId=8175861_1(8175862) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_1 uid=23340 usec=125 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_2(unfulfilled) for JobId=8175861_2(8175863) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_2 uid=23340 usec=115 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_3(unfulfilled) for JobId=8175861_3(8175864) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_3 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_4(unfulfilled) for JobId=8175861_4(8175865) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_4 uid=23340 usec=113 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_5(unfulfilled) for JobId=8175861_5(8175866) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_5 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_6(unfulfilled) for JobId=8175861_6(8175867) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_6 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_7(unfulfilled) for JobId=8175861_7(8175868) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_7 uid=23340 usec=114 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_8(unfulfilled) for JobId=8175861_8(8175869) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_8 uid=23340 usec=119 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_9(unfulfilled) for JobId=8175861_9(8175870) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_9 uid=23340 usec=114 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_10(unfulfilled) for JobId=8175861_10(8175871) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_10 uid=23340 usec=112 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_11(unfulfilled) for JobId=8175861_11(8175872) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_11 uid=23340 usec=108 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_12(unfulfilled) for JobId=8175861_12(8175873) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_12 uid=23340 usec=132 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_13(unfulfilled) for JobId=8175861_13(8175874) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_13 uid=23340 usec=139 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_14(unfulfilled) for JobId=8175861_14(8175875) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_14 uid=23340 usec=115 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_15(unfulfilled) for JobId=8175861_15(8175876) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_15 uid=23340 usec=115 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_16(unfulfilled) for JobId=8175861_16(8175877) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_16 uid=23340 usec=111 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_17(unfulfilled) for JobId=8175861_17(8175878) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_17 uid=23340 usec=110 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_18(unfulfilled) for JobId=8175861_18(8175879) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_18 uid=23340 usec=113 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_19(unfulfilled) for JobId=8175861_19(8175880) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_19 uid=23340 usec=150 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_20(unfulfilled) for JobId=8175861_20(8175881) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_20 uid=23340 usec=133 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_21(unfulfilled) for JobId=8175861_21(8175882) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_21 uid=23340 usec=119 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_22(unfulfilled) for JobId=8175861_22(8175883) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_22 uid=23340 usec=124 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_23(unfulfilled) for JobId=8175861_23(8175884) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_23 uid=23340 usec=118 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_24(unfulfilled) for JobId=8175861_24(8175885) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_24 uid=23340 usec=113 Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_25(unfulfilled) for JobId=8175861_25(8175886) Nov 05 12:22:02 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_25 uid=23340 usec=114 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_26(unfulfilled) for JobId=8175861_26(8175887) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_26 uid=23340 usec=17551 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_27(unfulfilled) for JobId=8175861_27(8175888) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_27 uid=23340 usec=141 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_28(unfulfilled) for JobId=8175861_28(8175889) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_28 uid=23340 usec=122 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_29(unfulfilled) for JobId=8175861_29(8175890) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_29 uid=23340 usec=230 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_30(unfulfilled) for JobId=8175861_30(8175891) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_30 uid=23340 usec=115 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_31(unfulfilled) for JobId=8175861_31(8175892) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_31 uid=23340 usec=116 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_32(unfulfilled) for JobId=8175861_32(8175893) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_32 uid=23340 usec=118 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_33(unfulfilled) for JobId=8175861_33(8175894) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_33 uid=23340 usec=115 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_34(unfulfilled) for JobId=8175861_34(8175895) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_34 uid=23340 usec=119 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_35(unfulfilled) for JobId=8175861_35(8175896) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_35 uid=23340 usec=119 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_36(unfulfilled) for JobId=8175861_36(8175897) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_36 uid=23340 usec=176 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_37(unfulfilled) for JobId=8175861_37(8175898) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_37 uid=23340 usec=121 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_38(unfulfilled) for JobId=8175861_38(8175899) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_38 uid=23340 usec=123 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_39(unfulfilled) for JobId=8175861_39(8175900) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_39 uid=23340 usec=231 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_40(unfulfilled) for JobId=8175861_40(8175901) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_40 uid=23340 usec=188 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_41(unfulfilled) for JobId=8175861_41(8175902) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_41 uid=23340 usec=110 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_42(unfulfilled) for JobId=8175861_42(8175903) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_42 uid=23340 usec=37521 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_43(unfulfilled) for JobId=8175861_43(8175904) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_43 uid=23340 usec=147 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_44(unfulfilled) for JobId=8175861_44(8175905) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_44 uid=23340 usec=121 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_45(unfulfilled) for JobId=8175861_45(8175906) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_45 uid=23340 usec=121 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_46(unfulfilled) for JobId=8175861_46(8175907) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_46 uid=23340 usec=110 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_186(8170770) WEXITSTATUS 0 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_186(8170770) done Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_47(unfulfilled) for JobId=8175861_47(8175908) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_47 uid=23340 usec=123 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_48(unfulfilled) for JobId=8175861_48(8175909) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_48 uid=23340 usec=118 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_49(unfulfilled) for JobId=8175861_49(8175910) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_49 uid=23340 usec=115 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_50(unfulfilled) for JobId=8175861_50(8175911) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_50 uid=23340 usec=119 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_51(unfulfilled) for JobId=8175861_51(8175912) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_51 uid=23340 usec=110 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_52(unfulfilled) for JobId=8175861_52(8175913) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_52 uid=23340 usec=110 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_53(unfulfilled) for JobId=8175861_53(8175914) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_53 uid=23340 usec=109 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_54(unfulfilled) for JobId=8175861_54(8175915) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_54 uid=23340 usec=118 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_55(unfulfilled) for JobId=8175861_55(8175916) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_55 uid=23340 usec=114 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_56(unfulfilled) for JobId=8175861_56(8175917) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_56 uid=23340 usec=111 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_57(unfulfilled) for JobId=8175861_57(8175918) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_57 uid=23340 usec=117 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_58(unfulfilled) for JobId=8175861_58(8175919) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_58 uid=23340 usec=113 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_59(unfulfilled) for JobId=8175861_59(8175920) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_59 uid=23340 usec=109 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_60(unfulfilled) for JobId=8175861_60(8175921) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_60 uid=23340 usec=113 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_61(unfulfilled) for JobId=8175861_61(8175922) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_61 uid=23340 usec=118 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_62(unfulfilled) for JobId=8175861_62(8175923) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_62 uid=23340 usec=113 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_63(unfulfilled) for JobId=8175861_63(8175924) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_63 uid=23340 usec=201 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_64(unfulfilled) for JobId=8175861_64(8175925) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_64 uid=23340 usec=191 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_65(unfulfilled) for JobId=8175861_65(8175926) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_65 uid=23340 usec=133 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_66(unfulfilled) for JobId=8175861_66(8175927) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_66 uid=23340 usec=117 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_67(unfulfilled) for JobId=8175861_67(8175928) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_67 uid=23340 usec=118 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_68(unfulfilled) for JobId=8175861_68(8175929) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_68 uid=23340 usec=129 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_69(unfulfilled) for JobId=8175861_69(8175930) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_69 uid=23340 usec=126 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_70(unfulfilled) for JobId=8175861_70(8175931) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_70 uid=23340 usec=124 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_71(unfulfilled) for JobId=8175861_71(8175932) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_71 uid=23340 usec=117 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_72(unfulfilled) for JobId=8175861_72(8175933) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_72 uid=23340 usec=138 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_73(unfulfilled) for JobId=8175861_73(8175934) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_73 uid=23340 usec=129 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_74(unfulfilled) for JobId=8175861_74(8175935) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_74 uid=23340 usec=122 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_75(unfulfilled) for JobId=8175861_75(8175936) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_75 uid=23340 usec=166 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_76(unfulfilled) for JobId=8175861_76(8175937) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_76 uid=23340 usec=120 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_77(unfulfilled) for JobId=8175861_77(8175938) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_77 uid=23340 usec=116 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_78(unfulfilled) for JobId=8175861_78(8175939) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_78 uid=23340 usec=115 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_79(unfulfilled) for JobId=8175861_79(8175940) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_79 uid=23340 usec=127 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_80(unfulfilled) for JobId=8175861_80(8175941) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_80 uid=23340 usec=133 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_81(unfulfilled) for JobId=8175861_81(8175942) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_81 uid=23340 usec=119 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_82(unfulfilled) for JobId=8175861_82(8175943) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_82 uid=23340 usec=127 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_83(unfulfilled) for JobId=8175861_83(8175944) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_83 uid=23340 usec=123 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_84(unfulfilled) for JobId=8175861_84(8175945) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_84 uid=23340 usec=122 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_85(unfulfilled) for JobId=8175861_85(8175946) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_85 uid=23340 usec=118 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_86(unfulfilled) for JobId=8175861_86(8175947) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_86 uid=23340 usec=148 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_87(unfulfilled) for JobId=8175861_87(8175948) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_87 uid=23340 usec=117 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_88(unfulfilled) for JobId=8175861_88(8175949) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_88 uid=23340 usec=121 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_89(unfulfilled) for JobId=8175861_89(8175950) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_89 uid=23340 usec=322 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_90(unfulfilled) for JobId=8175861_90(8175951) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_90 uid=23340 usec=196 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_91(unfulfilled) for JobId=8175861_91(8175952) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_91 uid=23340 usec=117 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2451(8165734) WEXITSTATUS 0 Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2451(8165734) done Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_92(unfulfilled) for JobId=8175861_92(8175953) Nov 05 12:22:03 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_92 uid=23340 usec=185 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_93(unfulfilled) for JobId=8175861_93(8175954) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_93 uid=23340 usec=67706 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_94(unfulfilled) for JobId=8175861_94(8175955) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_94 uid=23340 usec=13081 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_95(unfulfilled) for JobId=8175861_95(8175956) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_95 uid=23340 usec=132 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175765_96(unfulfilled) for JobId=8175861_96(8175861) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175861_96 uid=23340 usec=105 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8175957 InitPrio=13477 usec=1325 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_1(unfulfilled) for JobId=8175957_1(8175958) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_1 uid=23340 usec=134 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_2(unfulfilled) for JobId=8175957_2(8175959) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_2 uid=23340 usec=119 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_3(unfulfilled) for JobId=8175957_3(8175960) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_3 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_4(unfulfilled) for JobId=8175957_4(8175961) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_4 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_5(unfulfilled) for JobId=8175957_5(8175962) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_5 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_6(unfulfilled) for JobId=8175957_6(8175963) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_6 uid=23340 usec=119 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_7(unfulfilled) for JobId=8175957_7(8175964) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_7 uid=23340 usec=115 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_8(unfulfilled) for JobId=8175957_8(8175965) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_8 uid=23340 usec=118 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_9(unfulfilled) for JobId=8175957_9(8175966) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_9 uid=23340 usec=115 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_10(unfulfilled) for JobId=8175957_10(8175967) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_10 uid=23340 usec=112 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_11(unfulfilled) for JobId=8175957_11(8175968) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_11 uid=23340 usec=111 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_12(unfulfilled) for JobId=8175957_12(8175969) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_12 uid=23340 usec=110 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_13(unfulfilled) for JobId=8175957_13(8175970) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_13 uid=23340 usec=129 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_14(unfulfilled) for JobId=8175957_14(8175971) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_14 uid=23340 usec=111 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_15(unfulfilled) for JobId=8175957_15(8175972) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_15 uid=23340 usec=119 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_16(unfulfilled) for JobId=8175957_16(8175973) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_16 uid=23340 usec=289 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_17(unfulfilled) for JobId=8175957_17(8175974) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_17 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_18(unfulfilled) for JobId=8175957_18(8175975) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_18 uid=23340 usec=113 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_19(unfulfilled) for JobId=8175957_19(8175976) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_19 uid=23340 usec=113 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_20(unfulfilled) for JobId=8175957_20(8175977) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_20 uid=23340 usec=183 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_21(unfulfilled) for JobId=8175957_21(8175978) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_21 uid=23340 usec=197 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_22(unfulfilled) for JobId=8175957_22(8175979) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_22 uid=23340 usec=112 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_23(unfulfilled) for JobId=8175957_23(8175980) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_23 uid=23340 usec=118 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_24(unfulfilled) for JobId=8175957_24(8175981) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_24 uid=23340 usec=115 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_25(unfulfilled) for JobId=8175957_25(8175982) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_25 uid=23340 usec=200 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_26(unfulfilled) for JobId=8175957_26(8175983) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_26 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_27(unfulfilled) for JobId=8175957_27(8175984) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_27 uid=23340 usec=117 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_28(unfulfilled) for JobId=8175957_28(8175985) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_28 uid=23340 usec=110 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_29(unfulfilled) for JobId=8175957_29(8175986) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_29 uid=23340 usec=111 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_30(unfulfilled) for JobId=8175957_30(8175987) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_30 uid=23340 usec=115 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_31(unfulfilled) for JobId=8175957_31(8175988) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_31 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_32(unfulfilled) for JobId=8175957_32(8175989) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_32 uid=23340 usec=315 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_33(unfulfilled) for JobId=8175957_33(8175990) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_33 uid=23340 usec=213 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_34(unfulfilled) for JobId=8175957_34(8175991) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_34 uid=23340 usec=119 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_35(unfulfilled) for JobId=8175957_35(8175992) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_35 uid=23340 usec=283 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_36(unfulfilled) for JobId=8175957_36(8175993) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_36 uid=23340 usec=115 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_37(unfulfilled) for JobId=8175957_37(8175994) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_37 uid=23340 usec=178 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_38(unfulfilled) for JobId=8175957_38(8175995) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_38 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_39(unfulfilled) for JobId=8175957_39(8175996) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_39 uid=23340 usec=110 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_40(unfulfilled) for JobId=8175957_40(8175997) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_40 uid=23340 usec=118 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_41(unfulfilled) for JobId=8175957_41(8175998) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_41 uid=23340 usec=113 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_42(unfulfilled) for JobId=8175957_42(8175999) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_42 uid=23340 usec=113 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_43(unfulfilled) for JobId=8175957_43(8176000) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_43 uid=23340 usec=118 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_44(unfulfilled) for JobId=8175957_44(8176001) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_44 uid=23340 usec=112 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_45(unfulfilled) for JobId=8175957_45(8176002) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_45 uid=23340 usec=40589 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_121(8170705) WEXITSTATUS 0 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_121(8170705) done Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_46(unfulfilled) for JobId=8175957_46(8176003) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_46 uid=23340 usec=164 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_47(unfulfilled) for JobId=8175957_47(8176004) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_47 uid=23340 usec=121 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_48(unfulfilled) for JobId=8175957_48(8176005) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_48 uid=23340 usec=117 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_49(unfulfilled) for JobId=8175957_49(8176006) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_49 uid=23340 usec=129 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_50(unfulfilled) for JobId=8175957_50(8176007) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_50 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_51(unfulfilled) for JobId=8175957_51(8176008) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_51 uid=23340 usec=116 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_52(unfulfilled) for JobId=8175957_52(8176009) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_52 uid=23340 usec=115 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_53(unfulfilled) for JobId=8175957_53(8176010) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_53 uid=23340 usec=115 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_54(unfulfilled) for JobId=8175957_54(8176011) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_54 uid=23340 usec=114 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_55(unfulfilled) for JobId=8175957_55(8176012) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_55 uid=23340 usec=119 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_56(unfulfilled) for JobId=8175957_56(8176013) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_56 uid=23340 usec=118 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_57(unfulfilled) for JobId=8175957_57(8176014) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_57 uid=23340 usec=117 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_58(unfulfilled) for JobId=8175957_58(8176015) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_58 uid=23340 usec=111 Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_59(unfulfilled) for JobId=8175957_59(8176016) Nov 05 12:22:04 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_59 uid=23340 usec=120 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_60(unfulfilled) for JobId=8175957_60(8176017) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_60 uid=23340 usec=18245 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_61(unfulfilled) for JobId=8175957_61(8176018) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_61 uid=23340 usec=196 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_664(8171637) WEXITSTATUS 0 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_664(8171637) done Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_62(unfulfilled) for JobId=8175957_62(8176019) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_62 uid=23340 usec=167 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_63(unfulfilled) for JobId=8175957_63(8176020) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_63 uid=23340 usec=153 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_64(unfulfilled) for JobId=8175957_64(8176021) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_64 uid=23340 usec=129 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_65(unfulfilled) for JobId=8175957_65(8176022) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_65 uid=23340 usec=156 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_66(unfulfilled) for JobId=8175957_66(8176023) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_66 uid=23340 usec=159 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_67(unfulfilled) for JobId=8175957_67(8176024) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_67 uid=23340 usec=129 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_68(unfulfilled) for JobId=8175957_68(8176025) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_68 uid=23340 usec=132 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_750(8171805) WEXITSTATUS 0 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_750(8171805) done Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_69(unfulfilled) for JobId=8175957_69(8176026) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_69 uid=23340 usec=139 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_70(unfulfilled) for JobId=8175957_70(8176027) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_70 uid=23340 usec=131 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_71(unfulfilled) for JobId=8175957_71(8176028) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_71 uid=23340 usec=130 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_72(unfulfilled) for JobId=8175957_72(8176029) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_72 uid=23340 usec=127 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_73(unfulfilled) for JobId=8175957_73(8176030) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_73 uid=23340 usec=126 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_74(unfulfilled) for JobId=8175957_74(8176031) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_74 uid=23340 usec=123 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_251(8170895) WEXITSTATUS 0 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_251(8170895) done Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_75(unfulfilled) for JobId=8175957_75(8176032) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_75 uid=23340 usec=132 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_76(unfulfilled) for JobId=8175957_76(8176033) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_76 uid=23340 usec=121 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_77(unfulfilled) for JobId=8175957_77(8176034) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_77 uid=23340 usec=129 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_78(unfulfilled) for JobId=8175957_78(8176035) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_78 uid=23340 usec=120 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_79(unfulfilled) for JobId=8175957_79(8176036) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_79 uid=23340 usec=126 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_80(unfulfilled) for JobId=8175957_80(8176037) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_80 uid=23340 usec=125 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_748(8171803) WEXITSTATUS 0 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_748(8171803) done Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_81(unfulfilled) for JobId=8175957_81(8176038) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_81 uid=23340 usec=137 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_82(unfulfilled) for JobId=8175957_82(8176039) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_82 uid=23340 usec=126 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_83(unfulfilled) for JobId=8175957_83(8176040) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_83 uid=23340 usec=124 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_84(unfulfilled) for JobId=8175957_84(8176041) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_84 uid=23340 usec=153 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_85(unfulfilled) for JobId=8175957_85(8176042) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_85 uid=23340 usec=125 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_86(unfulfilled) for JobId=8175957_86(8176043) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_86 uid=23340 usec=118 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_87(unfulfilled) for JobId=8175957_87(8176044) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_87 uid=23340 usec=166 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_88(unfulfilled) for JobId=8175957_88(8176045) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_88 uid=23340 usec=126 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_89(unfulfilled) for JobId=8175957_89(8176046) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_89 uid=23340 usec=120 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_90(unfulfilled) for JobId=8175957_90(8176047) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_90 uid=23340 usec=119 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_91(unfulfilled) for JobId=8175957_91(8176048) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_91 uid=23340 usec=241 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_92(unfulfilled) for JobId=8175957_92(8176049) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_92 uid=23340 usec=117 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_93(unfulfilled) for JobId=8175957_93(8176050) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_93 uid=23340 usec=119 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_94(unfulfilled) for JobId=8175957_94(8176051) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_94 uid=23340 usec=290 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_95(unfulfilled) for JobId=8175957_95(8176052) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_95 uid=23340 usec=45148 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175861_96(unfulfilled) for JobId=8175957_96(8175957) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8175957_96 uid=23340 usec=136 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176053 InitPrio=13477 usec=2262 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_1(unfulfilled) for JobId=8176053_1(8176054) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_1 uid=23340 usec=151 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_2(unfulfilled) for JobId=8176053_2(8176055) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_2 uid=23340 usec=123 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_3(unfulfilled) for JobId=8176053_3(8176056) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_3 uid=23340 usec=124 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_4(unfulfilled) for JobId=8176053_4(8176057) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_4 uid=23340 usec=122 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_5(unfulfilled) for JobId=8176053_5(8176058) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_5 uid=23340 usec=126 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_6(unfulfilled) for JobId=8176053_6(8176059) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_6 uid=23340 usec=118 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_7(unfulfilled) for JobId=8176053_7(8176060) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_7 uid=23340 usec=117 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_8(unfulfilled) for JobId=8176053_8(8176061) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_8 uid=23340 usec=116 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_9(unfulfilled) for JobId=8176053_9(8176062) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_9 uid=23340 usec=123 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_10(unfulfilled) for JobId=8176053_10(8176063) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_10 uid=23340 usec=119 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_11(unfulfilled) for JobId=8176053_11(8176064) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_11 uid=23340 usec=120 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_12(unfulfilled) for JobId=8176053_12(8176065) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_12 uid=23340 usec=126 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_13(unfulfilled) for JobId=8176053_13(8176066) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_13 uid=23340 usec=124 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_14(unfulfilled) for JobId=8176053_14(8176067) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_14 uid=23340 usec=119 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_15(unfulfilled) for JobId=8176053_15(8176068) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_15 uid=23340 usec=119 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_16(unfulfilled) for JobId=8176053_16(8176069) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_16 uid=23340 usec=134 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_17(unfulfilled) for JobId=8176053_17(8176070) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_17 uid=23340 usec=284 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_18(unfulfilled) for JobId=8176053_18(8176071) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_18 uid=23340 usec=119 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_19(unfulfilled) for JobId=8176053_19(8176072) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_19 uid=23340 usec=133 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_20(unfulfilled) for JobId=8176053_20(8176073) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_20 uid=23340 usec=125 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_21(unfulfilled) for JobId=8176053_21(8176074) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_21 uid=23340 usec=119 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_22(unfulfilled) for JobId=8176053_22(8176075) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_22 uid=23340 usec=210 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_23(unfulfilled) for JobId=8176053_23(8176076) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_23 uid=23340 usec=116 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_24(unfulfilled) for JobId=8176053_24(8176077) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_24 uid=23340 usec=122 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_25(unfulfilled) for JobId=8176053_25(8176078) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_25 uid=23340 usec=113 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_26(unfulfilled) for JobId=8176053_26(8176079) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_26 uid=23340 usec=178 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_27(unfulfilled) for JobId=8176053_27(8176080) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_27 uid=23340 usec=124 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_28(unfulfilled) for JobId=8176053_28(8176081) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_28 uid=23340 usec=145 Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_29(unfulfilled) for JobId=8176053_29(8176082) Nov 05 12:22:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_29 uid=23340 usec=125 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_30(unfulfilled) for JobId=8176053_30(8176083) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_30 uid=23340 usec=127 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_31(unfulfilled) for JobId=8176053_31(8176084) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_31 uid=23340 usec=12907 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_32(unfulfilled) for JobId=8176053_32(8176085) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_32 uid=23340 usec=122 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_33(unfulfilled) for JobId=8176053_33(8176086) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_33 uid=23340 usec=118 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_34(unfulfilled) for JobId=8176053_34(8176087) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_34 uid=23340 usec=311 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_35(unfulfilled) for JobId=8176053_35(8176088) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_35 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_36(unfulfilled) for JobId=8176053_36(8176089) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_36 uid=23340 usec=199 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_37(unfulfilled) for JobId=8176053_37(8176090) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_37 uid=23340 usec=109 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_38(unfulfilled) for JobId=8176053_38(8176091) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_38 uid=23340 usec=134 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_39(unfulfilled) for JobId=8176053_39(8176092) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_39 uid=23340 usec=121 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_40(unfulfilled) for JobId=8176053_40(8176093) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_40 uid=23340 usec=125 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_41(unfulfilled) for JobId=8176053_41(8176094) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_41 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_42(unfulfilled) for JobId=8176053_42(8176095) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_42 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_43(unfulfilled) for JobId=8176053_43(8176096) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_43 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_44(unfulfilled) for JobId=8176053_44(8176097) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_44 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_45(unfulfilled) for JobId=8176053_45(8176098) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_45 uid=23340 usec=107 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_46(unfulfilled) for JobId=8176053_46(8176099) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_46 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_47(unfulfilled) for JobId=8176053_47(8176100) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_47 uid=23340 usec=107 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_48(unfulfilled) for JobId=8176053_48(8176101) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_48 uid=23340 usec=132 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_49(unfulfilled) for JobId=8176053_49(8176102) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_49 uid=23340 usec=31606 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_50(unfulfilled) for JobId=8176053_50(8176103) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_50 uid=23340 usec=139 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_51(unfulfilled) for JobId=8176053_51(8176104) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_51 uid=23340 usec=116 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_52(unfulfilled) for JobId=8176053_52(8176105) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_52 uid=23340 usec=119 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_53(unfulfilled) for JobId=8176053_53(8176106) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_53 uid=23340 usec=113 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_54(unfulfilled) for JobId=8176053_54(8176107) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_54 uid=23340 usec=111 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_55(unfulfilled) for JobId=8176053_55(8176108) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_55 uid=23340 usec=135 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_56(unfulfilled) for JobId=8176053_56(8176109) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_56 uid=23340 usec=128 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_57(unfulfilled) for JobId=8176053_57(8176110) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_57 uid=23340 usec=123 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_58(unfulfilled) for JobId=8176053_58(8176111) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_58 uid=23340 usec=112 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_59(unfulfilled) for JobId=8176053_59(8176112) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_59 uid=23340 usec=114 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_60(unfulfilled) for JobId=8176053_60(8176113) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_60 uid=23340 usec=115 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_61(unfulfilled) for JobId=8176053_61(8176114) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_61 uid=23340 usec=109 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_62(unfulfilled) for JobId=8176053_62(8176115) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_62 uid=23340 usec=113 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_63(unfulfilled) for JobId=8176053_63(8176116) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_63 uid=23340 usec=111 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_64(unfulfilled) for JobId=8176053_64(8176117) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_64 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_65(unfulfilled) for JobId=8176053_65(8176118) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_65 uid=23340 usec=141 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_66(unfulfilled) for JobId=8176053_66(8176119) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_66 uid=23340 usec=121 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_67(unfulfilled) for JobId=8176053_67(8176120) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_67 uid=23340 usec=120 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_68(unfulfilled) for JobId=8176053_68(8176121) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_68 uid=23340 usec=110 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_69(unfulfilled) for JobId=8176053_69(8176122) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_69 uid=23340 usec=111 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_70(unfulfilled) for JobId=8176053_70(8176123) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_70 uid=23340 usec=109 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_71(unfulfilled) for JobId=8176053_71(8176124) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_71 uid=23340 usec=105 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_72(unfulfilled) for JobId=8176053_72(8176125) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_72 uid=23340 usec=177 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_73(unfulfilled) for JobId=8176053_73(8176126) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_73 uid=23340 usec=119 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_74(unfulfilled) for JobId=8176053_74(8176127) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_74 uid=23340 usec=113 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_75(unfulfilled) for JobId=8176053_75(8176128) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_75 uid=23340 usec=119 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_76(unfulfilled) for JobId=8176053_76(8176129) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_76 uid=23340 usec=243 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_77(unfulfilled) for JobId=8176053_77(8176130) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_77 uid=23340 usec=111 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_78(unfulfilled) for JobId=8176053_78(8176131) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_78 uid=23340 usec=218 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_79(unfulfilled) for JobId=8176053_79(8176132) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_79 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_80(unfulfilled) for JobId=8176053_80(8176133) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_80 uid=23340 usec=111 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_81(unfulfilled) for JobId=8176053_81(8176134) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_81 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_82(unfulfilled) for JobId=8176053_82(8176135) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_82 uid=23340 usec=114 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_83(unfulfilled) for JobId=8176053_83(8176136) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_83 uid=23340 usec=137 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_84(unfulfilled) for JobId=8176053_84(8176137) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_84 uid=23340 usec=119 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_85(unfulfilled) for JobId=8176053_85(8176138) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_85 uid=23340 usec=115 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_86(unfulfilled) for JobId=8176053_86(8176139) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_86 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_87(unfulfilled) for JobId=8176053_87(8176140) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_87 uid=23340 usec=111 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_88(unfulfilled) for JobId=8176053_88(8176141) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_88 uid=23340 usec=110 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_89(unfulfilled) for JobId=8176053_89(8176142) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_89 uid=23340 usec=113 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_90(unfulfilled) for JobId=8176053_90(8176143) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_90 uid=23340 usec=133 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_91(unfulfilled) for JobId=8176053_91(8176144) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_91 uid=23340 usec=114 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_92(unfulfilled) for JobId=8176053_92(8176145) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_92 uid=23340 usec=111 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_93(unfulfilled) for JobId=8176053_93(8176146) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_93 uid=23340 usec=108 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_94(unfulfilled) for JobId=8176053_94(8176147) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_94 uid=23340 usec=107 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_95(unfulfilled) for JobId=8176053_95(8176148) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_95 uid=23340 usec=223 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8175957_96(unfulfilled) for JobId=8176053_96(8176053) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176053_96 uid=23340 usec=98 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176149 InitPrio=13477 usec=2323 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_1(unfulfilled) for JobId=8176149_1(8176150) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_1 uid=23340 usec=129 Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_2(unfulfilled) for JobId=8176149_2(8176151) Nov 05 12:22:06 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_2 uid=23340 usec=126 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_3(unfulfilled) for JobId=8176149_3(8176152) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_3 uid=23340 usec=101462 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_4(unfulfilled) for JobId=8176149_4(8176153) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_4 uid=23340 usec=33462 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_5(unfulfilled) for JobId=8176149_5(8176154) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_5 uid=23340 usec=146 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_6(unfulfilled) for JobId=8176149_6(8176155) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_6 uid=23340 usec=121 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_7(unfulfilled) for JobId=8176149_7(8176156) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_7 uid=23340 usec=119 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_8(unfulfilled) for JobId=8176149_8(8176157) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_8 uid=23340 usec=139 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_9(unfulfilled) for JobId=8176149_9(8176158) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_9 uid=23340 usec=127 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_10(unfulfilled) for JobId=8176149_10(8176159) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_10 uid=23340 usec=116 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_11(unfulfilled) for JobId=8176149_11(8176160) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_11 uid=23340 usec=117 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_12(unfulfilled) for JobId=8176149_12(8176161) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_12 uid=23340 usec=116 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_13(unfulfilled) for JobId=8176149_13(8176162) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_13 uid=23340 usec=116 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_14(unfulfilled) for JobId=8176149_14(8176163) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_14 uid=23340 usec=117 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_15(unfulfilled) for JobId=8176149_15(8176164) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_15 uid=23340 usec=116 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_16(unfulfilled) for JobId=8176149_16(8176165) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_16 uid=23340 usec=114 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_17(unfulfilled) for JobId=8176149_17(8176166) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_17 uid=23340 usec=124 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_357(8171076) WEXITSTATUS 0 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_357(8171076) done Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_18(unfulfilled) for JobId=8176149_18(8176167) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_18 uid=23340 usec=184 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_19(unfulfilled) for JobId=8176149_19(8176168) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_19 uid=23340 usec=159 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_20(unfulfilled) for JobId=8176149_20(8176169) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_20 uid=23340 usec=177 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_21(unfulfilled) for JobId=8176149_21(8176170) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_21 uid=23340 usec=156 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_22(unfulfilled) for JobId=8176149_22(8176171) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_22 uid=23340 usec=145 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_23(unfulfilled) for JobId=8176149_23(8176172) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_23 uid=23340 usec=156 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_24(unfulfilled) for JobId=8176149_24(8176173) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_24 uid=23340 usec=141 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_25(unfulfilled) for JobId=8176149_25(8176174) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_25 uid=23340 usec=138 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_26(unfulfilled) for JobId=8176149_26(8176175) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_26 uid=23340 usec=292 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_27(unfulfilled) for JobId=8176149_27(8176176) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_27 uid=23340 usec=133 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_28(unfulfilled) for JobId=8176149_28(8176177) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_28 uid=23340 usec=138 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_29(unfulfilled) for JobId=8176149_29(8176178) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_29 uid=23340 usec=136 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_30(unfulfilled) for JobId=8176149_30(8176179) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_30 uid=23340 usec=220 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_31(unfulfilled) for JobId=8176149_31(8176180) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_31 uid=23340 usec=130 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_32(unfulfilled) for JobId=8176149_32(8176181) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_32 uid=23340 usec=129 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_33(unfulfilled) for JobId=8176149_33(8176182) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_33 uid=23340 usec=124 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_34(unfulfilled) for JobId=8176149_34(8176183) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_34 uid=23340 usec=113 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_35(unfulfilled) for JobId=8176149_35(8176184) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_35 uid=23340 usec=113 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_36(unfulfilled) for JobId=8176149_36(8176185) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_36 uid=23340 usec=157 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_37(unfulfilled) for JobId=8176149_37(8176186) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_37 uid=23340 usec=115 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_38(unfulfilled) for JobId=8176149_38(8176187) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_38 uid=23340 usec=145 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_39(unfulfilled) for JobId=8176149_39(8176188) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_39 uid=23340 usec=114 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_40(unfulfilled) for JobId=8176149_40(8176189) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_40 uid=23340 usec=113 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_41(unfulfilled) for JobId=8176149_41(8176190) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_41 uid=23340 usec=145 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_42(unfulfilled) for JobId=8176149_42(8176191) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_42 uid=23340 usec=113 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_43(unfulfilled) for JobId=8176149_43(8176192) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_43 uid=23340 usec=114 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_44(unfulfilled) for JobId=8176149_44(8176193) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_44 uid=23340 usec=120 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_45(unfulfilled) for JobId=8176149_45(8176194) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_45 uid=23340 usec=249 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_46(unfulfilled) for JobId=8176149_46(8176195) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_46 uid=23340 usec=115 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_47(unfulfilled) for JobId=8176149_47(8176196) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_47 uid=23340 usec=291 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_48(unfulfilled) for JobId=8176149_48(8176197) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_48 uid=23340 usec=181 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_49(unfulfilled) for JobId=8176149_49(8176198) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_49 uid=23340 usec=44255 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_50(unfulfilled) for JobId=8176149_50(8176199) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_50 uid=23340 usec=144 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_51(unfulfilled) for JobId=8176149_51(8176200) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_51 uid=23340 usec=136 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_52(unfulfilled) for JobId=8176149_52(8176201) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_52 uid=23340 usec=123 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_53(unfulfilled) for JobId=8176149_53(8176202) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_53 uid=23340 usec=121 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_54(unfulfilled) for JobId=8176149_54(8176203) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_54 uid=23340 usec=115 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_55(unfulfilled) for JobId=8176149_55(8176204) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_55 uid=23340 usec=113 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_56(unfulfilled) for JobId=8176149_56(8176205) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_56 uid=23340 usec=116 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_57(unfulfilled) for JobId=8176149_57(8176206) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_57 uid=23340 usec=117 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_58(unfulfilled) for JobId=8176149_58(8176207) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_58 uid=23340 usec=131 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_59(unfulfilled) for JobId=8176149_59(8176208) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_59 uid=23340 usec=121 Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_60(unfulfilled) for JobId=8176149_60(8176209) Nov 05 12:22:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_60 uid=23340 usec=117 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_61(unfulfilled) for JobId=8176149_61(8176210) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_61 uid=23340 usec=46785 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_62(unfulfilled) for JobId=8176149_62(8176211) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_62 uid=23340 usec=195 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_63(unfulfilled) for JobId=8176149_63(8176212) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_63 uid=23340 usec=126 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_64(unfulfilled) for JobId=8176149_64(8176213) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_64 uid=23340 usec=121 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_65(unfulfilled) for JobId=8176149_65(8176214) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_65 uid=23340 usec=127 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_66(unfulfilled) for JobId=8176149_66(8176215) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_66 uid=23340 usec=122 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_67(unfulfilled) for JobId=8176149_67(8176216) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_67 uid=23340 usec=153 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_68(unfulfilled) for JobId=8176149_68(8176217) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_68 uid=23340 usec=164 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_69(unfulfilled) for JobId=8176149_69(8176218) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_69 uid=23340 usec=148 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_70(unfulfilled) for JobId=8176149_70(8176219) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_70 uid=23340 usec=125 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_71(unfulfilled) for JobId=8176149_71(8176220) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_71 uid=23340 usec=292 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_72(unfulfilled) for JobId=8176149_72(8176221) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_72 uid=23340 usec=114 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_73(unfulfilled) for JobId=8176149_73(8176222) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_73 uid=23340 usec=279 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_74(unfulfilled) for JobId=8176149_74(8176223) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_74 uid=23340 usec=122 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_75(unfulfilled) for JobId=8176149_75(8176224) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_75 uid=23340 usec=161 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_76(unfulfilled) for JobId=8176149_76(8176225) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_76 uid=23340 usec=116 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_77(unfulfilled) for JobId=8176149_77(8176226) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_77 uid=23340 usec=118 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_78(unfulfilled) for JobId=8176149_78(8176227) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_78 uid=23340 usec=115 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_79(unfulfilled) for JobId=8176149_79(8176228) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_79 uid=23340 usec=118 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_80(unfulfilled) for JobId=8176149_80(8176229) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_80 uid=23340 usec=117 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_81(unfulfilled) for JobId=8176149_81(8176230) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_81 uid=23340 usec=198 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_82(unfulfilled) for JobId=8176149_82(8176231) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_82 uid=23340 usec=120 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_83(unfulfilled) for JobId=8176149_83(8176232) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_83 uid=23340 usec=121 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_84(unfulfilled) for JobId=8176149_84(8176233) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_84 uid=23340 usec=119 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_85(unfulfilled) for JobId=8176149_85(8176234) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_85 uid=23340 usec=136 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_86(unfulfilled) for JobId=8176149_86(8176235) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_86 uid=23340 usec=117 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_87(unfulfilled) for JobId=8176149_87(8176236) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_87 uid=23340 usec=153 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_88(unfulfilled) for JobId=8176149_88(8176237) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_88 uid=23340 usec=167 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_89(unfulfilled) for JobId=8176149_89(8176238) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_89 uid=23340 usec=116 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_90(unfulfilled) for JobId=8176149_90(8176239) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_90 uid=23340 usec=124 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_91(unfulfilled) for JobId=8176149_91(8176240) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_91 uid=23340 usec=123 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_92(unfulfilled) for JobId=8176149_92(8176241) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_92 uid=23340 usec=120 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_93(unfulfilled) for JobId=8176149_93(8176242) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_93 uid=23340 usec=116 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_94(unfulfilled) for JobId=8176149_94(8176243) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_94 uid=23340 usec=118 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_95(unfulfilled) for JobId=8176149_95(8176244) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_95 uid=23340 usec=124 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176053_96(unfulfilled) for JobId=8176149_96(8176149) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176149_96 uid=23340 usec=108 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176245 InitPrio=13477 usec=2759 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_1(unfulfilled) for JobId=8176245_1(8176246) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_1 uid=23340 usec=163 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_2(unfulfilled) for JobId=8176245_2(8176247) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_2 uid=23340 usec=127 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_3(unfulfilled) for JobId=8176245_3(8176248) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_3 uid=23340 usec=127 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_4(unfulfilled) for JobId=8176245_4(8176249) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_4 uid=23340 usec=120 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_5(unfulfilled) for JobId=8176245_5(8176250) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_5 uid=23340 usec=119 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_6(unfulfilled) for JobId=8176245_6(8176251) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_6 uid=23340 usec=117 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_7(unfulfilled) for JobId=8176245_7(8176252) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_7 uid=23340 usec=119 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_621(8171594) WEXITSTATUS 0 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_621(8171594) done Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_8(unfulfilled) for JobId=8176245_8(8176253) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_8 uid=23340 usec=149 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_9(unfulfilled) for JobId=8176245_9(8176254) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_9 uid=23340 usec=128 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_10(unfulfilled) for JobId=8176245_10(8176255) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_10 uid=23340 usec=125 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_11(unfulfilled) for JobId=8176245_11(8176256) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_11 uid=23340 usec=159 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_12(unfulfilled) for JobId=8176245_12(8176257) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_12 uid=23340 usec=141 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_13(unfulfilled) for JobId=8176245_13(8176258) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_13 uid=23340 usec=134 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_14(unfulfilled) for JobId=8176245_14(8176259) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_14 uid=23340 usec=135 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_15(unfulfilled) for JobId=8176245_15(8176260) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_15 uid=23340 usec=132 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_16(unfulfilled) for JobId=8176245_16(8176261) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_16 uid=23340 usec=133 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_17(unfulfilled) for JobId=8176245_17(8176262) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_17 uid=23340 usec=131 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_18(unfulfilled) for JobId=8176245_18(8176263) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_18 uid=23340 usec=121 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_19(unfulfilled) for JobId=8176245_19(8176264) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_19 uid=23340 usec=129 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_20(unfulfilled) for JobId=8176245_20(8176265) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_20 uid=23340 usec=124 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_21(unfulfilled) for JobId=8176245_21(8176266) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_21 uid=23340 usec=127 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_22(unfulfilled) for JobId=8176245_22(8176267) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_22 uid=23340 usec=126 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_23(unfulfilled) for JobId=8176245_23(8176268) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_23 uid=23340 usec=134 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_24(unfulfilled) for JobId=8176245_24(8176269) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_24 uid=23340 usec=128 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_25(unfulfilled) for JobId=8176245_25(8176270) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_25 uid=23340 usec=132 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_26(unfulfilled) for JobId=8176245_26(8176271) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_26 uid=23340 usec=230 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_27(unfulfilled) for JobId=8176245_27(8176272) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_27 uid=23340 usec=133 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_28(unfulfilled) for JobId=8176245_28(8176273) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_28 uid=23340 usec=132 Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_29(unfulfilled) for JobId=8176245_29(8176274) Nov 05 12:22:08 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_29 uid=23340 usec=129 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3374(8135173) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2650(8153037) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2673(8153060) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2690(8153077) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2703(8153090) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2705(8153092) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2711(8153098) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2714(8153101) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2728(8153115) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2732(8153119) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2735(8153122) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_30(unfulfilled) for JobId=8176245_30(8176275) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_30 uid=23340 usec=13438 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_31(unfulfilled) for JobId=8176245_31(8176276) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_31 uid=23340 usec=156 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_32(unfulfilled) for JobId=8176245_32(8176277) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_32 uid=23340 usec=138 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_33(unfulfilled) for JobId=8176245_33(8176278) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_33 uid=23340 usec=137 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_34(unfulfilled) for JobId=8176245_34(8176279) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_34 uid=23340 usec=169 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_35(unfulfilled) for JobId=8176245_35(8176280) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_35 uid=23340 usec=126 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_36(unfulfilled) for JobId=8176245_36(8176281) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_36 uid=23340 usec=123 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_37(unfulfilled) for JobId=8176245_37(8176282) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_37 uid=23340 usec=146 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_38(unfulfilled) for JobId=8176245_38(8176283) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_38 uid=23340 usec=124 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_39(unfulfilled) for JobId=8176245_39(8176284) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_39 uid=23340 usec=128 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_40(unfulfilled) for JobId=8176245_40(8176285) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_40 uid=23340 usec=124 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_41(unfulfilled) for JobId=8176245_41(8176286) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_41 uid=23340 usec=146 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_42(unfulfilled) for JobId=8176245_42(8176287) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_42 uid=23340 usec=231 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_43(unfulfilled) for JobId=8176245_43(8176288) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_43 uid=23340 usec=129 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_44(unfulfilled) for JobId=8176245_44(8176289) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_44 uid=23340 usec=126 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_45(unfulfilled) for JobId=8176245_45(8176290) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_45 uid=23340 usec=188 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_742(8171797) WEXITSTATUS 0 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_742(8171797) done Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_46(unfulfilled) for JobId=8176245_46(8176291) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_46 uid=23340 usec=138 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_47(unfulfilled) for JobId=8176245_47(8176292) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_47 uid=23340 usec=122 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_48(unfulfilled) for JobId=8176245_48(8176293) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_48 uid=23340 usec=126 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_49(unfulfilled) for JobId=8176245_49(8176294) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_49 uid=23340 usec=123 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_50(unfulfilled) for JobId=8176245_50(8176295) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_50 uid=23340 usec=125 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_51(unfulfilled) for JobId=8176245_51(8176296) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_51 uid=23340 usec=44270 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_52(unfulfilled) for JobId=8176245_52(8176297) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_52 uid=23340 usec=156 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_53(unfulfilled) for JobId=8176245_53(8176298) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_53 uid=23340 usec=128 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_54(unfulfilled) for JobId=8176245_54(8176299) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_54 uid=23340 usec=123 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_55(unfulfilled) for JobId=8176245_55(8176300) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_55 uid=23340 usec=123 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_56(unfulfilled) for JobId=8176245_56(8176301) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_56 uid=23340 usec=121 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_57(unfulfilled) for JobId=8176245_57(8176302) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_57 uid=23340 usec=124 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_162(8170746) WEXITSTATUS 0 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_162(8170746) done Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_58(unfulfilled) for JobId=8176245_58(8176303) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_58 uid=23340 usec=131 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_59(unfulfilled) for JobId=8176245_59(8176304) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_59 uid=23340 usec=127 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_60(unfulfilled) for JobId=8176245_60(8176305) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_60 uid=23340 usec=128 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_61(unfulfilled) for JobId=8176245_61(8176306) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_61 uid=23340 usec=122 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_62(unfulfilled) for JobId=8176245_62(8176307) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_62 uid=23340 usec=127 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_63(unfulfilled) for JobId=8176245_63(8176308) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_63 uid=23340 usec=121 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_64(unfulfilled) for JobId=8176245_64(8176309) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_64 uid=23340 usec=143 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_633(8171606) WEXITSTATUS 0 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_633(8171606) done Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_65(unfulfilled) for JobId=8176245_65(8176310) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_65 uid=23340 usec=137 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_66(unfulfilled) for JobId=8176245_66(8176311) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_66 uid=23340 usec=131 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_67(unfulfilled) for JobId=8176245_67(8176312) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_67 uid=23340 usec=121 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_68(unfulfilled) for JobId=8176245_68(8176313) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_68 uid=23340 usec=120 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_69(unfulfilled) for JobId=8176245_69(8176314) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_69 uid=23340 usec=125 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_70(unfulfilled) for JobId=8176245_70(8176315) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_70 uid=23340 usec=204 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_71(unfulfilled) for JobId=8176245_71(8176316) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_71 uid=23340 usec=124 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_72(unfulfilled) for JobId=8176245_72(8176317) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_72 uid=23340 usec=207 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_73(unfulfilled) for JobId=8176245_73(8176318) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_73 uid=23340 usec=166 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_74(unfulfilled) for JobId=8176245_74(8176319) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_74 uid=23340 usec=242 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_75(unfulfilled) for JobId=8176245_75(8176320) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_75 uid=23340 usec=118 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_76(unfulfilled) for JobId=8176245_76(8176321) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_76 uid=23340 usec=120 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_77(unfulfilled) for JobId=8176245_77(8176322) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_77 uid=23340 usec=118 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_78(unfulfilled) for JobId=8176245_78(8176323) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_78 uid=23340 usec=119 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_79(unfulfilled) for JobId=8176245_79(8176324) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_79 uid=23340 usec=121 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_80(unfulfilled) for JobId=8176245_80(8176325) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_80 uid=23340 usec=119 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_81(unfulfilled) for JobId=8176245_81(8176326) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_81 uid=23340 usec=122 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_82(unfulfilled) for JobId=8176245_82(8176327) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_82 uid=23340 usec=227 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_83(unfulfilled) for JobId=8176245_83(8176328) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_83 uid=23340 usec=115 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_84(unfulfilled) for JobId=8176245_84(8176329) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_84 uid=23340 usec=155 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_85(unfulfilled) for JobId=8176245_85(8176330) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_85 uid=23340 usec=218 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_86(unfulfilled) for JobId=8176245_86(8176331) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_86 uid=23340 usec=186 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_87(unfulfilled) for JobId=8176245_87(8176332) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_87 uid=23340 usec=138 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_88(unfulfilled) for JobId=8176245_88(8176333) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_88 uid=23340 usec=138 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_89(unfulfilled) for JobId=8176245_89(8176334) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_89 uid=23340 usec=198 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_90(unfulfilled) for JobId=8176245_90(8176335) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_90 uid=23340 usec=263 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_91(unfulfilled) for JobId=8176245_91(8176336) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_91 uid=23340 usec=121 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_92(unfulfilled) for JobId=8176245_92(8176337) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_92 uid=23340 usec=124 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_93(unfulfilled) for JobId=8176245_93(8176338) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_93 uid=23340 usec=172 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_94(unfulfilled) for JobId=8176245_94(8176339) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_94 uid=23340 usec=122 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_95(unfulfilled) for JobId=8176245_95(8176340) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_95 uid=23340 usec=122 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176149_96(unfulfilled) for JobId=8176245_96(8176245) Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176245_96 uid=23340 usec=108 Nov 05 12:22:09 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176341 InitPrio=13477 usec=2440 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_1(unfulfilled) for JobId=8176341_1(8176342) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_1 uid=23340 usec=20476 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_2(unfulfilled) for JobId=8176341_2(8176343) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_2 uid=23340 usec=370 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_3(unfulfilled) for JobId=8176341_3(8176344) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_3 uid=23340 usec=136 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_4(unfulfilled) for JobId=8176341_4(8176345) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_4 uid=23340 usec=39122 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_5(unfulfilled) for JobId=8176341_5(8176346) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_5 uid=23340 usec=152 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_6(unfulfilled) for JobId=8176341_6(8176347) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_6 uid=23340 usec=146 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_7(unfulfilled) for JobId=8176341_7(8176348) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_7 uid=23340 usec=136 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_8(unfulfilled) for JobId=8176341_8(8176349) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_8 uid=23340 usec=129 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_9(unfulfilled) for JobId=8176341_9(8176350) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_9 uid=23340 usec=125 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_10(unfulfilled) for JobId=8176341_10(8176351) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_10 uid=23340 usec=135 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_11(unfulfilled) for JobId=8176341_11(8176352) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_11 uid=23340 usec=132 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_12(unfulfilled) for JobId=8176341_12(8176353) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_12 uid=23340 usec=131 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_13(unfulfilled) for JobId=8176341_13(8176354) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_13 uid=23340 usec=140 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_14(unfulfilled) for JobId=8176341_14(8176355) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_14 uid=23340 usec=136 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_15(unfulfilled) for JobId=8176341_15(8176356) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_15 uid=23340 usec=125 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_16(unfulfilled) for JobId=8176341_16(8176357) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_16 uid=23340 usec=131 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_17(unfulfilled) for JobId=8176341_17(8176358) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_17 uid=23340 usec=125 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_18(unfulfilled) for JobId=8176341_18(8176359) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_18 uid=23340 usec=136 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_19(unfulfilled) for JobId=8176341_19(8176360) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_19 uid=23340 usec=126 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_20(unfulfilled) for JobId=8176341_20(8176361) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_20 uid=23340 usec=118 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_21(unfulfilled) for JobId=8176341_21(8176362) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_21 uid=23340 usec=121 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_22(unfulfilled) for JobId=8176341_22(8176363) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_22 uid=23340 usec=120 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_23(unfulfilled) for JobId=8176341_23(8176364) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_23 uid=23340 usec=125 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_24(unfulfilled) for JobId=8176341_24(8176365) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_24 uid=23340 usec=138 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_25(unfulfilled) for JobId=8176341_25(8176366) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_25 uid=23340 usec=116 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_26(unfulfilled) for JobId=8176341_26(8176367) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_26 uid=23340 usec=113 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_27(unfulfilled) for JobId=8176341_27(8176368) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_27 uid=23340 usec=144 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_28(unfulfilled) for JobId=8176341_28(8176369) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_28 uid=23340 usec=128 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_29(unfulfilled) for JobId=8176341_29(8176370) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_29 uid=23340 usec=119 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_30(unfulfilled) for JobId=8176341_30(8176371) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_30 uid=23340 usec=340 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_31(unfulfilled) for JobId=8176341_31(8176372) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_31 uid=23340 usec=116 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_32(unfulfilled) for JobId=8176341_32(8176373) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_32 uid=23340 usec=118 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_33(unfulfilled) for JobId=8176341_33(8176374) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_33 uid=23340 usec=197 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_34(unfulfilled) for JobId=8176341_34(8176375) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_34 uid=23340 usec=114 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_35(unfulfilled) for JobId=8176341_35(8176376) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_35 uid=23340 usec=210 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_36(unfulfilled) for JobId=8176341_36(8176377) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_36 uid=23340 usec=120 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_37(unfulfilled) for JobId=8176341_37(8176378) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_37 uid=23340 usec=118 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_38(unfulfilled) for JobId=8176341_38(8176379) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_38 uid=23340 usec=112 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_39(unfulfilled) for JobId=8176341_39(8176380) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_39 uid=23340 usec=113 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_40(unfulfilled) for JobId=8176341_40(8176381) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_40 uid=23340 usec=118 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_41(unfulfilled) for JobId=8176341_41(8176382) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_41 uid=23340 usec=115 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_42(unfulfilled) for JobId=8176341_42(8176383) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_42 uid=23340 usec=112 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_43(unfulfilled) for JobId=8176341_43(8176384) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_43 uid=23340 usec=313 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_69(8170609) WEXITSTATUS 0 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_69(8170609) done Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_44(unfulfilled) for JobId=8176341_44(8176385) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_44 uid=23340 usec=122 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_45(unfulfilled) for JobId=8176341_45(8176386) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_45 uid=23340 usec=245 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_46(unfulfilled) for JobId=8176341_46(8176387) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_46 uid=23340 usec=116 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_47(unfulfilled) for JobId=8176341_47(8176388) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_47 uid=23340 usec=115 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_48(unfulfilled) for JobId=8176341_48(8176389) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_48 uid=23340 usec=117 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_49(unfulfilled) for JobId=8176341_49(8176390) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_49 uid=23340 usec=122 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_50(unfulfilled) for JobId=8176341_50(8176391) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_50 uid=23340 usec=120 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_51(unfulfilled) for JobId=8176341_51(8176392) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_51 uid=23340 usec=120 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_52(unfulfilled) for JobId=8176341_52(8176393) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_52 uid=23340 usec=136 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_53(unfulfilled) for JobId=8176341_53(8176394) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_53 uid=23340 usec=128 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_54(unfulfilled) for JobId=8176341_54(8176395) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_54 uid=23340 usec=123 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_55(unfulfilled) for JobId=8176341_55(8176396) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_55 uid=23340 usec=203 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_56(unfulfilled) for JobId=8176341_56(8176397) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_56 uid=23340 usec=41307 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_57(unfulfilled) for JobId=8176341_57(8176398) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_57 uid=23340 usec=191 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_58(unfulfilled) for JobId=8176341_58(8176399) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_58 uid=23340 usec=135 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_59(unfulfilled) for JobId=8176341_59(8176400) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_59 uid=23340 usec=129 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_60(unfulfilled) for JobId=8176341_60(8176401) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_60 uid=23340 usec=125 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_61(unfulfilled) for JobId=8176341_61(8176402) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_61 uid=23340 usec=143 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_62(unfulfilled) for JobId=8176341_62(8176403) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_62 uid=23340 usec=142 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_63(unfulfilled) for JobId=8176341_63(8176404) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_63 uid=23340 usec=141 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_64(unfulfilled) for JobId=8176341_64(8176405) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_64 uid=23340 usec=135 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_65(unfulfilled) for JobId=8176341_65(8176406) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_65 uid=23340 usec=130 Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_66(unfulfilled) for JobId=8176341_66(8176407) Nov 05 12:22:10 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_66 uid=23340 usec=131 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_67(unfulfilled) for JobId=8176341_67(8176408) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_67 uid=23340 usec=47839 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_68(unfulfilled) for JobId=8176341_68(8176409) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_68 uid=23340 usec=214 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_69(unfulfilled) for JobId=8176341_69(8176410) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_69 uid=23340 usec=128 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_70(unfulfilled) for JobId=8176341_70(8176411) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_70 uid=23340 usec=119 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_71(unfulfilled) for JobId=8176341_71(8176412) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_71 uid=23340 usec=145 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_72(unfulfilled) for JobId=8176341_72(8176413) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_72 uid=23340 usec=184 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_73(unfulfilled) for JobId=8176341_73(8176414) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_73 uid=23340 usec=119 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_74(unfulfilled) for JobId=8176341_74(8176415) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_74 uid=23340 usec=122 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_75(unfulfilled) for JobId=8176341_75(8176416) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_75 uid=23340 usec=150 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_76(unfulfilled) for JobId=8176341_76(8176417) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_76 uid=23340 usec=129 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_77(unfulfilled) for JobId=8176341_77(8176418) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_77 uid=23340 usec=122 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_78(unfulfilled) for JobId=8176341_78(8176419) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_78 uid=23340 usec=122 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_79(unfulfilled) for JobId=8176341_79(8176420) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_79 uid=23340 usec=121 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_80(unfulfilled) for JobId=8176341_80(8176421) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_80 uid=23340 usec=123 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_81(unfulfilled) for JobId=8176341_81(8176422) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_81 uid=23340 usec=135 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_82(unfulfilled) for JobId=8176341_82(8176423) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_82 uid=23340 usec=131 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_83(unfulfilled) for JobId=8176341_83(8176424) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_83 uid=23340 usec=120 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_84(unfulfilled) for JobId=8176341_84(8176425) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_84 uid=23340 usec=126 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_85(unfulfilled) for JobId=8176341_85(8176426) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_85 uid=23340 usec=119 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_86(unfulfilled) for JobId=8176341_86(8176427) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_86 uid=23340 usec=120 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_87(unfulfilled) for JobId=8176341_87(8176428) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_87 uid=23340 usec=119 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_88(unfulfilled) for JobId=8176341_88(8176429) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_88 uid=23340 usec=115 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_89(unfulfilled) for JobId=8176341_89(8176430) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_89 uid=23340 usec=114 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_90(unfulfilled) for JobId=8176341_90(8176431) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_90 uid=23340 usec=130 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_91(unfulfilled) for JobId=8176341_91(8176432) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_91 uid=23340 usec=132 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_92(unfulfilled) for JobId=8176341_92(8176433) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_92 uid=23340 usec=129 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_93(unfulfilled) for JobId=8176341_93(8176434) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_93 uid=23340 usec=115 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_94(unfulfilled) for JobId=8176341_94(8176435) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_94 uid=23340 usec=117 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_95(unfulfilled) for JobId=8176341_95(8176436) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_95 uid=23340 usec=127 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176245_96(unfulfilled) for JobId=8176341_96(8176341) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176341_96 uid=23340 usec=113 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176437 InitPrio=13477 usec=2306 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_1(unfulfilled) for JobId=8176437_1(8176438) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_1 uid=23340 usec=139 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_2(unfulfilled) for JobId=8176437_2(8176439) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_2 uid=23340 usec=169 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_3(unfulfilled) for JobId=8176437_3(8176440) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_3 uid=23340 usec=120 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_4(unfulfilled) for JobId=8176437_4(8176441) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_4 uid=23340 usec=123 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_5(unfulfilled) for JobId=8176437_5(8176442) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_5 uid=23340 usec=118 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_6(unfulfilled) for JobId=8176437_6(8176443) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_6 uid=23340 usec=197 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_7(unfulfilled) for JobId=8176437_7(8176444) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_7 uid=23340 usec=123 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_8(unfulfilled) for JobId=8176437_8(8176445) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_8 uid=23340 usec=38489 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_9(unfulfilled) for JobId=8176437_9(8176446) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_9 uid=23340 usec=151 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_10(unfulfilled) for JobId=8176437_10(8176447) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_10 uid=23340 usec=125 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_11(unfulfilled) for JobId=8176437_11(8176448) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_11 uid=23340 usec=143 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_12(unfulfilled) for JobId=8176437_12(8176449) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_12 uid=23340 usec=131 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_13(unfulfilled) for JobId=8176437_13(8176450) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_13 uid=23340 usec=125 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_14(unfulfilled) for JobId=8176437_14(8176451) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_14 uid=23340 usec=135 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_15(unfulfilled) for JobId=8176437_15(8176452) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_15 uid=23340 usec=127 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_16(unfulfilled) for JobId=8176437_16(8176453) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_16 uid=23340 usec=128 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_17(unfulfilled) for JobId=8176437_17(8176454) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_17 uid=23340 usec=128 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_18(unfulfilled) for JobId=8176437_18(8176455) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_18 uid=23340 usec=138 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_19(unfulfilled) for JobId=8176437_19(8176456) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_19 uid=23340 usec=140 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_20(unfulfilled) for JobId=8176437_20(8176457) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_20 uid=23340 usec=129 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_21(unfulfilled) for JobId=8176437_21(8176458) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_21 uid=23340 usec=130 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_22(unfulfilled) for JobId=8176437_22(8176459) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_22 uid=23340 usec=128 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_23(unfulfilled) for JobId=8176437_23(8176460) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_23 uid=23340 usec=122 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_24(unfulfilled) for JobId=8176437_24(8176461) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_24 uid=23340 usec=126 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_25(unfulfilled) for JobId=8176437_25(8176462) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_25 uid=23340 usec=126 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_26(unfulfilled) for JobId=8176437_26(8176463) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_26 uid=23340 usec=126 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_27(unfulfilled) for JobId=8176437_27(8176464) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_27 uid=23340 usec=138 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_28(unfulfilled) for JobId=8176437_28(8176465) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_28 uid=23340 usec=128 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_29(unfulfilled) for JobId=8176437_29(8176466) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_29 uid=23340 usec=129 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_30(unfulfilled) for JobId=8176437_30(8176467) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_30 uid=23340 usec=127 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_31(unfulfilled) for JobId=8176437_31(8176468) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_31 uid=23340 usec=122 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_32(unfulfilled) for JobId=8176437_32(8176469) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_32 uid=23340 usec=131 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_33(unfulfilled) for JobId=8176437_33(8176470) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_33 uid=23340 usec=130 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_34(unfulfilled) for JobId=8176437_34(8176471) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_34 uid=23340 usec=135 Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_35(unfulfilled) for JobId=8176437_35(8176472) Nov 05 12:22:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_35 uid=23340 usec=131 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_36(unfulfilled) for JobId=8176437_36(8176473) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_36 uid=23340 usec=96662 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_37(unfulfilled) for JobId=8176437_37(8176474) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_37 uid=23340 usec=159 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_38(unfulfilled) for JobId=8176437_38(8176475) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_38 uid=23340 usec=130 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_39(unfulfilled) for JobId=8176437_39(8176476) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_39 uid=23340 usec=122 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_40(unfulfilled) for JobId=8176437_40(8176477) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_40 uid=23340 usec=122 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_41(unfulfilled) for JobId=8176437_41(8176478) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_41 uid=23340 usec=133 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_42(unfulfilled) for JobId=8176437_42(8176479) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_42 uid=23340 usec=123 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_43(unfulfilled) for JobId=8176437_43(8176480) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_43 uid=23340 usec=131 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_44(unfulfilled) for JobId=8176437_44(8176481) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_44 uid=23340 usec=112 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_45(unfulfilled) for JobId=8176437_45(8176482) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_45 uid=23340 usec=153 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_46(unfulfilled) for JobId=8176437_46(8176483) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_46 uid=23340 usec=131 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_47(unfulfilled) for JobId=8176437_47(8176484) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_47 uid=23340 usec=115 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_48(unfulfilled) for JobId=8176437_48(8176485) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_48 uid=23340 usec=301 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_49(unfulfilled) for JobId=8176437_49(8176486) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_49 uid=23340 usec=299 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_50(unfulfilled) for JobId=8176437_50(8176487) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_50 uid=23340 usec=180 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_51(unfulfilled) for JobId=8176437_51(8176488) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_51 uid=23340 usec=108 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_52(unfulfilled) for JobId=8176437_52(8176489) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_52 uid=23340 usec=135 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_53(unfulfilled) for JobId=8176437_53(8176490) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_53 uid=23340 usec=35162 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_54(unfulfilled) for JobId=8176437_54(8176491) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_54 uid=23340 usec=142 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_55(unfulfilled) for JobId=8176437_55(8176492) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_55 uid=23340 usec=115 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_56(unfulfilled) for JobId=8176437_56(8176493) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_56 uid=23340 usec=115 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_57(unfulfilled) for JobId=8176437_57(8176494) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_57 uid=23340 usec=109 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_58(unfulfilled) for JobId=8176437_58(8176495) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_58 uid=23340 usec=112 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_59(unfulfilled) for JobId=8176437_59(8176496) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_59 uid=23340 usec=113 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_60(unfulfilled) for JobId=8176437_60(8176497) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_60 uid=23340 usec=133 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_61(unfulfilled) for JobId=8176437_61(8176498) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_61 uid=23340 usec=130 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_62(unfulfilled) for JobId=8176437_62(8176499) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_62 uid=23340 usec=114 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_63(unfulfilled) for JobId=8176437_63(8176500) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_63 uid=23340 usec=112 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_64(unfulfilled) for JobId=8176437_64(8176501) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_64 uid=23340 usec=112 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_65(unfulfilled) for JobId=8176437_65(8176502) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_65 uid=23340 usec=111 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_66(unfulfilled) for JobId=8176437_66(8176503) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_66 uid=23340 usec=112 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_67(unfulfilled) for JobId=8176437_67(8176504) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_67 uid=23340 usec=130 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_68(unfulfilled) for JobId=8176437_68(8176505) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_68 uid=23340 usec=119 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_69(unfulfilled) for JobId=8176437_69(8176506) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_69 uid=23340 usec=119 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_70(unfulfilled) for JobId=8176437_70(8176507) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_70 uid=23340 usec=111 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_71(unfulfilled) for JobId=8176437_71(8176508) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_71 uid=23340 usec=118 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_72(unfulfilled) for JobId=8176437_72(8176509) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_72 uid=23340 usec=111 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_73(unfulfilled) for JobId=8176437_73(8176510) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_73 uid=23340 usec=112 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_74(unfulfilled) for JobId=8176437_74(8176511) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_74 uid=23340 usec=206 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_75(unfulfilled) for JobId=8176437_75(8176512) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_75 uid=23340 usec=186 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_76(unfulfilled) for JobId=8176437_76(8176513) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_76 uid=23340 usec=113 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_77(unfulfilled) for JobId=8176437_77(8176514) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_77 uid=23340 usec=140 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_78(unfulfilled) for JobId=8176437_78(8176515) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_78 uid=23340 usec=120 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_79(unfulfilled) for JobId=8176437_79(8176516) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_79 uid=23340 usec=111 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_80(unfulfilled) for JobId=8176437_80(8176517) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_80 uid=23340 usec=138 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_81(unfulfilled) for JobId=8176437_81(8176518) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_81 uid=23340 usec=171 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_82(unfulfilled) for JobId=8176437_82(8176519) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_82 uid=23340 usec=109 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_83(unfulfilled) for JobId=8176437_83(8176520) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_83 uid=23340 usec=115 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_84(unfulfilled) for JobId=8176437_84(8176521) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_84 uid=23340 usec=111 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_85(unfulfilled) for JobId=8176437_85(8176522) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_85 uid=23340 usec=108 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_86(unfulfilled) for JobId=8176437_86(8176523) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_86 uid=23340 usec=223 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_87(unfulfilled) for JobId=8176437_87(8176524) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_87 uid=23340 usec=128 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_88(unfulfilled) for JobId=8176437_88(8176525) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_88 uid=23340 usec=203 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_89(unfulfilled) for JobId=8176437_89(8176526) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_89 uid=23340 usec=109 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_90(unfulfilled) for JobId=8176437_90(8176527) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_90 uid=23340 usec=109 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_91(unfulfilled) for JobId=8176437_91(8176528) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_91 uid=23340 usec=108 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_92(unfulfilled) for JobId=8176437_92(8176529) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_92 uid=23340 usec=107 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_93(unfulfilled) for JobId=8176437_93(8176530) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_93 uid=23340 usec=107 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_94(unfulfilled) for JobId=8176437_94(8176531) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_94 uid=23340 usec=109 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_95(unfulfilled) for JobId=8176437_95(8176532) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_95 uid=23340 usec=111 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176341_96(unfulfilled) for JobId=8176437_96(8176437) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176437_96 uid=23340 usec=106 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176533 InitPrio=13477 usec=2363 Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_1(unfulfilled) for JobId=8176533_1(8176534) Nov 05 12:22:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_1 uid=23340 usec=181 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_2(unfulfilled) for JobId=8176533_2(8176535) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_2 uid=23340 usec=19884 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_3(unfulfilled) for JobId=8176533_3(8176536) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_3 uid=23340 usec=182 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_4(unfulfilled) for JobId=8176533_4(8176537) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_4 uid=23340 usec=126 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_5(unfulfilled) for JobId=8176533_5(8176538) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_5 uid=23340 usec=119 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_6(unfulfilled) for JobId=8176533_6(8176539) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_6 uid=23340 usec=120 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_7(unfulfilled) for JobId=8176533_7(8176540) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_7 uid=23340 usec=139 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_8(unfulfilled) for JobId=8176533_8(8176541) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_8 uid=23340 usec=32428 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_9(unfulfilled) for JobId=8176533_9(8176542) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_9 uid=23340 usec=172 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_240(8170884) WEXITSTATUS 0 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_240(8170884) done Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_10(unfulfilled) for JobId=8176533_10(8176543) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_10 uid=23340 usec=267 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_11(unfulfilled) for JobId=8176533_11(8176544) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_11 uid=23340 usec=122 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_12(unfulfilled) for JobId=8176533_12(8176545) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_12 uid=23340 usec=127 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_13(unfulfilled) for JobId=8176533_13(8176546) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_13 uid=23340 usec=121 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_14(unfulfilled) for JobId=8176533_14(8176547) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_14 uid=23340 usec=125 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_15(unfulfilled) for JobId=8176533_15(8176548) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_15 uid=23340 usec=122 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_16(unfulfilled) for JobId=8176533_16(8176549) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_16 uid=23340 usec=130 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_17(unfulfilled) for JobId=8176533_17(8176550) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_17 uid=23340 usec=126 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_18(unfulfilled) for JobId=8176533_18(8176551) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_18 uid=23340 usec=122 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_19(unfulfilled) for JobId=8176533_19(8176552) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_19 uid=23340 usec=121 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_20(unfulfilled) for JobId=8176533_20(8176553) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_20 uid=23340 usec=121 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_21(unfulfilled) for JobId=8176533_21(8176554) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_21 uid=23340 usec=122 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_22(unfulfilled) for JobId=8176533_22(8176555) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_22 uid=23340 usec=126 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_23(unfulfilled) for JobId=8176533_23(8176556) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_23 uid=23340 usec=148 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_24(unfulfilled) for JobId=8176533_24(8176557) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_24 uid=23340 usec=132 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_25(unfulfilled) for JobId=8176533_25(8176558) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_25 uid=23340 usec=122 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_26(unfulfilled) for JobId=8176533_26(8176559) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_26 uid=23340 usec=120 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_27(unfulfilled) for JobId=8176533_27(8176560) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_27 uid=23340 usec=117 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_28(unfulfilled) for JobId=8176533_28(8176561) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_28 uid=23340 usec=120 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_29(unfulfilled) for JobId=8176533_29(8176562) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_29 uid=23340 usec=213 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_30(unfulfilled) for JobId=8176533_30(8176563) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_30 uid=23340 usec=176 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_31(unfulfilled) for JobId=8176533_31(8176564) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_31 uid=23340 usec=129 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_32(unfulfilled) for JobId=8176533_32(8176565) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_32 uid=23340 usec=125 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_33(unfulfilled) for JobId=8176533_33(8176566) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_33 uid=23340 usec=243 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_34(unfulfilled) for JobId=8176533_34(8176567) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_34 uid=23340 usec=120 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_35(unfulfilled) for JobId=8176533_35(8176568) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_35 uid=23340 usec=119 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_36(unfulfilled) for JobId=8176533_36(8176569) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_36 uid=23340 usec=117 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_37(unfulfilled) for JobId=8176533_37(8176570) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_37 uid=23340 usec=148 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_38(unfulfilled) for JobId=8176533_38(8176571) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_38 uid=23340 usec=132 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_39(unfulfilled) for JobId=8176533_39(8176572) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_39 uid=23340 usec=123 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_40(unfulfilled) for JobId=8176533_40(8176573) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_40 uid=23340 usec=118 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_41(unfulfilled) for JobId=8176533_41(8176574) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_41 uid=23340 usec=119 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_42(unfulfilled) for JobId=8176533_42(8176575) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_42 uid=23340 usec=115 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_43(unfulfilled) for JobId=8176533_43(8176576) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_43 uid=23340 usec=117 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_44(unfulfilled) for JobId=8176533_44(8176577) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_44 uid=23340 usec=120 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_45(unfulfilled) for JobId=8176533_45(8176578) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_45 uid=23340 usec=118 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_46(unfulfilled) for JobId=8176533_46(8176579) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_46 uid=23340 usec=114 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_47(unfulfilled) for JobId=8176533_47(8176580) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_47 uid=23340 usec=147 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_48(unfulfilled) for JobId=8176533_48(8176581) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_48 uid=23340 usec=207 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_49(unfulfilled) for JobId=8176533_49(8176582) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_49 uid=23340 usec=117 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_50(unfulfilled) for JobId=8176533_50(8176583) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_50 uid=23340 usec=114 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_51(unfulfilled) for JobId=8176533_51(8176584) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_51 uid=23340 usec=114 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_52(unfulfilled) for JobId=8176533_52(8176585) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_52 uid=23340 usec=114 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_53(unfulfilled) for JobId=8176533_53(8176586) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_53 uid=23340 usec=114 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_54(unfulfilled) for JobId=8176533_54(8176587) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_54 uid=23340 usec=138 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_55(unfulfilled) for JobId=8176533_55(8176588) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_55 uid=23340 usec=136 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_56(unfulfilled) for JobId=8176533_56(8176589) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_56 uid=23340 usec=116 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_57(unfulfilled) for JobId=8176533_57(8176590) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_57 uid=23340 usec=256 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_58(unfulfilled) for JobId=8176533_58(8176591) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_58 uid=23340 usec=117 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_59(unfulfilled) for JobId=8176533_59(8176592) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_59 uid=23340 usec=188 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_60(unfulfilled) for JobId=8176533_60(8176593) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_60 uid=23340 usec=116 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_61(unfulfilled) for JobId=8176533_61(8176594) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_61 uid=23340 usec=227 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_62(unfulfilled) for JobId=8176533_62(8176595) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_62 uid=23340 usec=38171 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_63(unfulfilled) for JobId=8176533_63(8176596) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_63 uid=23340 usec=174 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_64(unfulfilled) for JobId=8176533_64(8176597) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_64 uid=23340 usec=137 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_65(unfulfilled) for JobId=8176533_65(8176598) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_65 uid=23340 usec=133 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_66(unfulfilled) for JobId=8176533_66(8176599) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_66 uid=23340 usec=122 Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_67(unfulfilled) for JobId=8176533_67(8176600) Nov 05 12:22:13 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_67 uid=23340 usec=120 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_68(unfulfilled) for JobId=8176533_68(8176601) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_68 uid=23340 usec=12589 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_69(unfulfilled) for JobId=8176533_69(8176602) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_69 uid=23340 usec=122 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_70(unfulfilled) for JobId=8176533_70(8176603) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_70 uid=23340 usec=145 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_71(unfulfilled) for JobId=8176533_71(8176604) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_71 uid=23340 usec=132 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_72(unfulfilled) for JobId=8176533_72(8176605) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_72 uid=23340 usec=128 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_73(unfulfilled) for JobId=8176533_73(8176606) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_73 uid=23340 usec=127 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_74(unfulfilled) for JobId=8176533_74(8176607) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_74 uid=23340 usec=126 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_75(unfulfilled) for JobId=8176533_75(8176608) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_75 uid=23340 usec=124 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_76(unfulfilled) for JobId=8176533_76(8176609) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_76 uid=23340 usec=127 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_77(unfulfilled) for JobId=8176533_77(8176610) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_77 uid=23340 usec=125 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_78(unfulfilled) for JobId=8176533_78(8176611) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_78 uid=23340 usec=124 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_79(unfulfilled) for JobId=8176533_79(8176612) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_79 uid=23340 usec=118 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_80(unfulfilled) for JobId=8176533_80(8176613) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_80 uid=23340 usec=121 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_81(unfulfilled) for JobId=8176533_81(8176614) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_81 uid=23340 usec=222 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_82(unfulfilled) for JobId=8176533_82(8176615) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_82 uid=23340 usec=132 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_83(unfulfilled) for JobId=8176533_83(8176616) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_83 uid=23340 usec=136 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_84(unfulfilled) for JobId=8176533_84(8176617) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_84 uid=23340 usec=153 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_85(unfulfilled) for JobId=8176533_85(8176618) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_85 uid=23340 usec=125 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_86(unfulfilled) for JobId=8176533_86(8176619) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_86 uid=23340 usec=120 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_87(unfulfilled) for JobId=8176533_87(8176620) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_87 uid=23340 usec=123 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_88(unfulfilled) for JobId=8176533_88(8176621) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_88 uid=23340 usec=163 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_89(unfulfilled) for JobId=8176533_89(8176622) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_89 uid=23340 usec=132 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_90(unfulfilled) for JobId=8176533_90(8176623) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_90 uid=23340 usec=133 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_91(unfulfilled) for JobId=8176533_91(8176624) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_91 uid=23340 usec=124 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_92(unfulfilled) for JobId=8176533_92(8176625) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_92 uid=23340 usec=181 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_93(unfulfilled) for JobId=8176533_93(8176626) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_93 uid=23340 usec=134 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_94(unfulfilled) for JobId=8176533_94(8176627) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_94 uid=23340 usec=119 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_95(unfulfilled) for JobId=8176533_95(8176628) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_95 uid=23340 usec=120 Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: sched: _update_job: setting dependency to afterany:8176437_96(unfulfilled) for JobId=8176533_96(8176533) Nov 05 12:22:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_job: complete JobId=8176533_96 uid=23340 usec=121 Nov 05 12:22:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_132(8170716) WEXITSTATUS 0 Nov 05 12:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_132(8170716) done Nov 05 12:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2307(8165193) WEXITSTATUS 0 Nov 05 12:22:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2307(8165193) done Nov 05 12:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2315(8165235) WEXITSTATUS 0 Nov 05 12:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2315(8165235) done Nov 05 12:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_183(8170767) WEXITSTATUS 0 Nov 05 12:22:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_183(8170767) done Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_178(8170762) WEXITSTATUS 0 Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_178(8170762) done Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_737(8171792) WEXITSTATUS 0 Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_737(8171792) done Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_221(8170865) WEXITSTATUS 0 Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_221(8170865) done Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_34(8170574) WEXITSTATUS 0 Nov 05 12:22:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_34(8170574) done Nov 05 12:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_141(8170725) WEXITSTATUS 0 Nov 05 12:22:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_141(8170725) done Nov 05 12:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_220(8170864) WEXITSTATUS 0 Nov 05 12:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_220(8170864) done Nov 05 12:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6791(8164945) WEXITSTATUS 1 Nov 05 12:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6791(8164945) done Nov 05 12:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_157(8170741) WEXITSTATUS 0 Nov 05 12:22:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_157(8170741) done Nov 05 12:22:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_726(8171781) WEXITSTATUS 0 Nov 05 12:22:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_726(8171781) done Nov 05 12:22:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_165(8170749) WEXITSTATUS 0 Nov 05 12:22:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_165(8170749) done Nov 05 12:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_709(8171764) WEXITSTATUS 0 Nov 05 12:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_709(8171764) done Nov 05 12:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_171(8170755) WEXITSTATUS 0 Nov 05 12:22:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_171(8170755) done Nov 05 12:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_169(8170753) WEXITSTATUS 0 Nov 05 12:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_169(8170753) done Nov 05 12:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_74(8170614) WEXITSTATUS 0 Nov 05 12:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_74(8170614) done Nov 05 12:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_192(8170776) WEXITSTATUS 0 Nov 05 12:22:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_192(8170776) done Nov 05 12:22:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_153(8170737) WEXITSTATUS 0 Nov 05 12:22:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_153(8170737) done Nov 05 12:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_724(8171779) WEXITSTATUS 0 Nov 05 12:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_724(8171779) done Nov 05 12:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_334(8171053) WEXITSTATUS 0 Nov 05 12:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_334(8171053) done Nov 05 12:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_155(8170739) WEXITSTATUS 0 Nov 05 12:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_155(8170739) done Nov 05 12:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_101(8170685) WEXITSTATUS 0 Nov 05 12:22:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_101(8170685) done Nov 05 12:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_166(8170750) WEXITSTATUS 0 Nov 05 12:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_166(8170750) done Nov 05 12:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_139(8170723) WEXITSTATUS 0 Nov 05 12:22:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_139(8170723) done Nov 05 12:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_135(8170719) WEXITSTATUS 0 Nov 05 12:22:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_135(8170719) done Nov 05 12:22:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_120(8170704) WEXITSTATUS 0 Nov 05 12:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_120(8170704) done Nov 05 12:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_131(8170715) WEXITSTATUS 0 Nov 05 12:22:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_131(8170715) done Nov 05 12:22:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_297(8170941) WEXITSTATUS 0 Nov 05 12:22:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_297(8170941) done Nov 05 12:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_798(8171865) WEXITSTATUS 0 Nov 05 12:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_798(8171865) done Nov 05 12:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_333(8171052) WEXITSTATUS 0 Nov 05 12:22:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_333(8171052) done Nov 05 12:22:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_138(8170722) WEXITSTATUS 0 Nov 05 12:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_138(8170722) done Nov 05 12:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7744(8168847) WEXITSTATUS 1 Nov 05 12:22:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7744(8168847) done Nov 05 12:22:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_184(8170768) WEXITSTATUS 0 Nov 05 12:22:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_184(8170768) done Nov 05 12:22:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_128(8170712) WEXITSTATUS 0 Nov 05 12:22:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_128(8170712) done Nov 05 12:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_71(8170611) WEXITSTATUS 0 Nov 05 12:22:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_71(8170611) done Nov 05 12:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_643(8171616) WEXITSTATUS 0 Nov 05 12:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_643(8171616) done Nov 05 12:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8049(8172430) WEXITSTATUS 1 Nov 05 12:22:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8049(8172430) done Nov 05 12:22:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3964(8146007) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:22:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_718(8171773) WEXITSTATUS 0 Nov 05 12:22:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_718(8171773) done Nov 05 12:22:39 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3964(8146007) Nov 05 12:22:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_172(8170756) WEXITSTATUS 0 Nov 05 12:22:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_172(8170756) done Nov 05 12:22:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_185(8170769) WEXITSTATUS 0 Nov 05 12:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_185(8170769) done Nov 05 12:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_191(8170775) WEXITSTATUS 0 Nov 05 12:22:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_191(8170775) done Nov 05 12:22:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_182(8170766) WEXITSTATUS 0 Nov 05 12:22:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_182(8170766) done Nov 05 12:22:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_677(8171650) WEXITSTATUS 0 Nov 05 12:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_677(8171650) done Nov 05 12:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_148(8170732) WEXITSTATUS 0 Nov 05 12:22:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_148(8170732) done Nov 05 12:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_678(8171651) WEXITSTATUS 0 Nov 05 12:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_678(8171651) done Nov 05 12:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2291(8165154) WEXITSTATUS 0 Nov 05 12:22:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2291(8165154) done Nov 05 12:22:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_136(8170720) WEXITSTATUS 0 Nov 05 12:22:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_136(8170720) done Nov 05 12:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_744(8171799) WEXITSTATUS 0 Nov 05 12:22:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_744(8171799) done Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173001 NodeList=r815u03n07 #CPUs=16 Partition=day Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173002 NodeList=r815u23n03 #CPUs=16 Partition=day Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173003 NodeList=r816u03n06 #CPUs=16 Partition=day Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175635_8(8176629) NodeList=r807u39n01 #CPUs=32 Partition=bigmem Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175586 NodeList=r815u09n11 #CPUs=8 Partition=scavenge Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175587 NodeList=r815u23n09 #CPUs=8 Partition=scavenge Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_193(8170777) WEXITSTATUS 0 Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_193(8170777) done Nov 05 12:22:55 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1600 Nov 05 12:22:56 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1600 Nov 05 12:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175563 WEXITSTATUS 0 Nov 05 12:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175563 done Nov 05 12:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175562 WEXITSTATUS 0 Nov 05 12:22:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175562 done Nov 05 12:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_247(8170891) WEXITSTATUS 0 Nov 05 12:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_247(8170891) done Nov 05 12:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175561 WEXITSTATUS 0 Nov 05 12:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175561 done Nov 05 12:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_784(8171850) WEXITSTATUS 0 Nov 05 12:22:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_784(8171850) done Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175560 WEXITSTATUS 0 Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175560 done Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175564 WEXITSTATUS 0 Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175564 done Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8056(8172494) WEXITSTATUS 1 Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8056(8172494) done Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175559 WEXITSTATUS 0 Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175559 done Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_8(8176629) WEXITSTATUS 0 Nov 05 12:22:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_8(8176629) done Nov 05 12:22:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:22:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7971(8170952) WEXITSTATUS 1 Nov 05 12:22:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7971(8170952) done Nov 05 12:23:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_743(8171798) WEXITSTATUS 0 Nov 05 12:23:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_743(8171798) done Nov 05 12:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_273(8170917) WEXITSTATUS 0 Nov 05 12:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_273(8170917) done Nov 05 12:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_367(8171086) WEXITSTATUS 0 Nov 05 12:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_367(8171086) done Nov 05 12:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_72(8170612) WEXITSTATUS 0 Nov 05 12:23:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_72(8170612) done Nov 05 12:23:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_681(8171654) WEXITSTATUS 0 Nov 05 12:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_681(8171654) done Nov 05 12:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175565 WEXITSTATUS 0 Nov 05 12:23:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175565 done Nov 05 12:23:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_164(8170748) WEXITSTATUS 0 Nov 05 12:23:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_164(8170748) done Nov 05 12:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_272(8170916) WEXITSTATUS 0 Nov 05 12:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_272(8170916) done Nov 05 12:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_364(8171083) WEXITSTATUS 0 Nov 05 12:23:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_364(8171083) done Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_359(8171078) WEXITSTATUS 0 Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_359(8171078) done Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_665(8171638) WEXITSTATUS 0 Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_665(8171638) done Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4033(8175641) WEXITSTATUS 0 Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4033(8175641) done Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_298(8170942) WEXITSTATUS 0 Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_298(8170942) done Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_188(8170772) WEXITSTATUS 0 Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_188(8170772) done Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_368(8171087) WEXITSTATUS 0 Nov 05 12:23:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_368(8171087) done Nov 05 12:23:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_168(8170752) WEXITSTATUS 0 Nov 05 12:23:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_168(8170752) done Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_358(8171077) WEXITSTATUS 0 Nov 05 12:23:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_358(8171077) done Nov 05 12:23:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8077(8172585) WEXITSTATUS 1 Nov 05 12:23:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8077(8172585) done Nov 05 12:23:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_337(8171056) WEXITSTATUS 0 Nov 05 12:23:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_337(8171056) done Nov 05 12:23:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_346(8171065) WEXITSTATUS 0 Nov 05 12:23:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_346(8171065) done Nov 05 12:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_347(8171066) WEXITSTATUS 0 Nov 05 12:23:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_347(8171066) done Nov 05 12:23:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_816(8171983) WEXITSTATUS 0 Nov 05 12:23:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_816(8171983) done Nov 05 12:23:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_484(8171280) WEXITSTATUS 0 Nov 05 12:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_484(8171280) done Nov 05 12:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_266(8170910) WEXITSTATUS 0 Nov 05 12:23:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_266(8170910) done Nov 05 12:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4034(8175652) WEXITSTATUS 0 Nov 05 12:23:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4034(8175652) done Nov 05 12:23:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_241(8170885) WEXITSTATUS 0 Nov 05 12:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_241(8170885) done Nov 05 12:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_804(8171871) WEXITSTATUS 0 Nov 05 12:23:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_804(8171871) done Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_710(8171765) WEXITSTATUS 0 Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_710(8171765) done Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_151(8170735) WEXITSTATUS 0 Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_151(8170735) done Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144140 WEXITSTATUS 0 Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144140 done Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_366(8171085) WEXITSTATUS 0 Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_366(8171085) done Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_341(8171060) WEXITSTATUS 0 Nov 05 12:23:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_341(8171060) done Nov 05 12:23:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175635_9(8176630) in bigmem on r807u39n01 Nov 05 12:23:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4036(8176631) in gpu on r4513u20n01 Nov 05 12:23:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4037(8176632) in gpu on r903u34n01 Nov 05 12:23:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4038(8176633) in gpu on r907u32n01 Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_152(8170736) WEXITSTATUS 0 Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_152(8170736) done Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_102(8170686) WEXITSTATUS 0 Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_102(8170686) done Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_365(8171084) WEXITSTATUS 0 Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_365(8171084) done Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_653(8171626) WEXITSTATUS 0 Nov 05 12:23:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_653(8171626) done Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175588 in scavenge on r815u15n11 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175589 in scavenge on r815u23n08 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175590 in scavenge on r816u03n10 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175591 in scavenge on r816u03n11 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175592 in scavenge on r816u03n11 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175593 in scavenge on r816u23n07 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175594 in scavenge on r816u23n09 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175595 in scavenge on r815u29n04 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175596 in scavenge on r816u15n04 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175597 in scavenge on r816u15n04 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175598 in scavenge on r816u15n07 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175599 in scavenge on r815u03n02 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175600 in scavenge on r815u03n08 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175601 in scavenge on r815u03n08 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175602 in scavenge on r815u09n02 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175603 in scavenge on r815u15n08 Nov 05 12:23:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175604 in scavenge on r815u15n08 Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_239(8170883) WEXITSTATUS 0 Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_239(8170883) done Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_352(8171071) WEXITSTATUS 0 Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_352(8171071) done Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_632(8171605) WEXITSTATUS 0 Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_632(8171605) done Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_339(8171058) WEXITSTATUS 0 Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_339(8171058) done Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_340(8171059) WEXITSTATUS 0 Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_340(8171059) done Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_103(8170687) WEXITSTATUS 0 Nov 05 12:23:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_103(8170687) done Nov 05 12:23:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_9(8176630) WEXITSTATUS 0 Nov 05 12:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175635_9(8176630) done Nov 05 12:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_361(8171080) WEXITSTATUS 0 Nov 05 12:23:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_361(8171080) done Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175566 WEXITSTATUS 0 Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175566 done Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_312(8171031) WEXITSTATUS 0 Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_312(8171031) done Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175567 WEXITSTATUS 0 Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175567 done Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_799(8171866) WEXITSTATUS 0 Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_799(8171866) done Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_369(8171088) WEXITSTATUS 0 Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_369(8171088) done Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_370(8171089) WEXITSTATUS 0 Nov 05 12:23:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_370(8171089) done Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_800(8171867) WEXITSTATUS 0 Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_800(8171867) done Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_227(8170871) WEXITSTATUS 0 Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_227(8170871) done Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_716(8171771) WEXITSTATUS 0 Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_716(8171771) done Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175573 WEXITSTATUS 0 Nov 05 12:23:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175573 done Nov 05 12:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175572 WEXITSTATUS 0 Nov 05 12:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175572 done Nov 05 12:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_331(8171050) WEXITSTATUS 0 Nov 05 12:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_331(8171050) done Nov 05 12:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175571 WEXITSTATUS 0 Nov 05 12:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175571 done Nov 05 12:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_786(8171852) WEXITSTATUS 0 Nov 05 12:23:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_786(8171852) done Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3885(8144950) in scavenge on r909u07n04 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3001(8128794) in scavenge on r914u05n01 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3890(8145069) in scavenge on r919u36n03 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1405(8123636) in scavenge on r805u06n01 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4173(8151033) in scavenge on r805u06n01 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_520(8113722) in scavenge on r805u23n01 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4195(8151840) in scavenge on r807u05n04 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2558(8127729) in scavenge on r807u08n04 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3835(8144449) in scavenge on r912u36n04 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4136(8149611) in scavenge on r913u05n01 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4151(8150167) in scavenge on r913u05n01 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1184(7572151) in scavenge on r806u28n01 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_694(7572863) in scavenge on r806u37n01 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_252(8170896) WEXITSTATUS 0 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_252(8170896) done Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_230(8170874) WEXITSTATUS 0 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_230(8170874) done Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_75(8170615) WEXITSTATUS 0 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_75(8170615) done Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_802(8171869) WEXITSTATUS 0 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_802(8171869) done Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175569 WEXITSTATUS 0 Nov 05 12:23:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175569 done Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175568 WEXITSTATUS 0 Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175568 done Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_106(8170690) WEXITSTATUS 0 Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_106(8170690) done Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175574 WEXITSTATUS 0 Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175574 done Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175570 WEXITSTATUS 0 Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175570 done Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175575 WEXITSTATUS 0 Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175575 done Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_105(8170689) WEXITSTATUS 0 Nov 05 12:23:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_105(8170689) done Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_1(8176634) in scavenge on r908u28n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_2(8176635) in scavenge on r908u33n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_3(8176636) in scavenge on r908u33n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_4(8176637) in scavenge on r908u37n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_5(8176638) in scavenge on r908u37n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_6(8176639) in scavenge on r908u37n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_7(8176640) in scavenge on r908u37n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_8(8176641) in scavenge on r908u37n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_9(8176642) in scavenge on r908u37n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_10(8176643) in scavenge on r908u37n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_11(8176644) in scavenge on r908u37n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_12(8176645) in scavenge on r908u39n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_13(8176646) in scavenge on r907u01n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_14(8176647) in scavenge on r907u01n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_15(8176648) in scavenge on r907u01n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_16(8176649) in scavenge on r907u01n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_17(8176650) in scavenge on r907u01n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_18(8176651) in scavenge on r907u01n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_19(8176652) in scavenge on r907u05n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_20(8176653) in scavenge on r907u05n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_21(8176654) in scavenge on r907u05n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_22(8176655) in scavenge on r907u05n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_23(8176656) in scavenge on r907u05n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_24(8176657) in scavenge on r907u05n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_25(8176658) in scavenge on r907u05n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_26(8176659) in scavenge on r907u05n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_27(8176660) in scavenge on r908u35n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_28(8176661) in scavenge on r908u35n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_29(8176662) in scavenge on r908u35n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_30(8176663) in scavenge on r908u35n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_31(8176664) in scavenge on r908u35n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_32(8176665) in scavenge on r908u35n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_33(8176666) in scavenge on r917u10n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_34(8176667) in scavenge on r801u11n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_35(8176668) in scavenge on r802u30n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_36(8176669) in scavenge on r802u30n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_37(8176670) in scavenge on r911u09n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_38(8176671) in scavenge on r911u09n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_39(8176672) in scavenge on r911u28n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_40(8176673) in scavenge on r911u28n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_41(8176674) in scavenge on r911u28n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_42(8176675) in scavenge on r911u28n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_43(8176676) in scavenge on r911u28n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_44(8176677) in scavenge on r912u01n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_45(8176678) in scavenge on r816u09n09 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_46(8176679) in scavenge on r815u03n10 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_47(8176680) in scavenge on r815u03n10 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_48(8176681) in scavenge on r815u09n10 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_49(8176682) in scavenge on r816u23n07 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_50(8176683) in scavenge on r816u23n08 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_51(8176684) in scavenge on r902u07n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_52(8176685) in scavenge on r902u07n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_53(8176686) in scavenge on r907u32n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_54(8176687) in scavenge on r906u28n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_55(8176688) in scavenge on r806u06n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_56(8176689) in scavenge on r906u41n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_57(8176690) in scavenge on r906u41n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_58(8176691) in scavenge on r904u04n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_59(8176692) in scavenge on r904u04n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_60(8176693) in scavenge on r904u04n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_61(8176694) in scavenge on r904u04n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_62(8176695) in scavenge on r805u06n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_63(8176696) in scavenge on r805u06n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_64(8176697) in scavenge on r805u11n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_65(8176698) in scavenge on r805u11n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_66(8176699) in scavenge on r805u23n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_67(8176700) in scavenge on r918u05n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_68(8176701) in scavenge on r918u09n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_69(8176702) in scavenge on r919u01n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_70(8176703) in scavenge on r919u01n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_71(8176704) in scavenge on r909u01n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_72(8176705) in scavenge on r909u01n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_73(8176706) in scavenge on r909u07n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_74(8176707) in scavenge on r909u10n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_75(8176708) in scavenge on r909u13n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_76(8176709) in scavenge on r904u10n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_77(8176710) in scavenge on r904u10n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_78(8176711) in scavenge on r914u01n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_79(8176712) in scavenge on r914u01n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_80(8176713) in scavenge on r914u05n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_81(8176714) in scavenge on r914u05n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_82(8176715) in scavenge on r807u03n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_83(8176716) in scavenge on r807u03n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_84(8176717) in scavenge on r807u05n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_85(8176718) in scavenge on r807u05n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_86(8176719) in scavenge on r807u05n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_87(8176720) in scavenge on r807u05n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_88(8176721) in scavenge on r807u05n03 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_89(8176722) in scavenge on r807u08n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_90(8176723) in scavenge on r807u08n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_91(8176724) in scavenge on r807u08n04 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_92(8176725) in scavenge on r807u11n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_93(8176726) in scavenge on r807u11n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_94(8176727) in scavenge on r807u11n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_95(8176728) in scavenge on r807u11n02 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8175668_96(8175668) in scavenge on r912u28n01 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2987(8176729) in scavenge on r815u15n09 Nov 05 12:23:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2988(8176730) in scavenge on r815u23n07 Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175635 uid 16603 Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_21(8175454) WEXITSTATUS 0 Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_21(8175454) done Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_631(8171604) WEXITSTATUS 0 Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_631(8171604) done Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_268(8170912) WEXITSTATUS 0 Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_268(8170912) done Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_644(8171617) WEXITSTATUS 0 Nov 05 12:23:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_644(8171617) done Nov 05 12:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_711(8171766) WEXITSTATUS 0 Nov 05 12:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_711(8171766) done Nov 05 12:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_371(8171090) WEXITSTATUS 0 Nov 05 12:23:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_371(8171090) done Nov 05 12:23:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_356(8171075) WEXITSTATUS 0 Nov 05 12:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_356(8171075) done Nov 05 12:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_354(8171073) WEXITSTATUS 0 Nov 05 12:23:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_354(8171073) done Nov 05 12:23:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_617(8171590) WEXITSTATUS 0 Nov 05 12:23:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_617(8171590) done Nov 05 12:23:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_174(8170758) WEXITSTATUS 0 Nov 05 12:23:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_174(8170758) done Nov 05 12:23:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4036(8176631) WEXITSTATUS 0 Nov 05 12:23:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4036(8176631) done Nov 05 12:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_336(8171055) WEXITSTATUS 0 Nov 05 12:23:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_336(8171055) done Nov 05 12:23:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_360(8171079) WEXITSTATUS 0 Nov 05 12:23:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_360(8171079) done Nov 05 12:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_51(8176684) WEXITSTATUS 1 Nov 05 12:23:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_51(8176684) done Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_52(8176685) WEXITSTATUS 1 Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_52(8176685) done Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_48(8176681) WEXITSTATUS 1 Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_48(8176681) done Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_45(8176678) WEXITSTATUS 1 Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_45(8176678) done Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_736(8171791) WEXITSTATUS 0 Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_736(8171791) done Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_50(8176683) WEXITSTATUS 1 Nov 05 12:23:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_50(8176683) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_49(8176682) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_49(8176682) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_785(8171851) WEXITSTATUS 0 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_785(8171851) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_47(8176680) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_47(8176680) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_46(8176679) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_46(8176679) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_16(8176649) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_16(8176649) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_15(8176648) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_15(8176648) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_23(8176656) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_23(8176656) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_24(8176657) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_24(8176657) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_18(8176651) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_18(8176651) done Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_17(8176650) WEXITSTATUS 1 Nov 05 12:23:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_17(8176650) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_21(8176654) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_21(8176654) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_10(8176643) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_10(8176643) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_11(8176644) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_11(8176644) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_20(8176653) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_20(8176653) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_22(8176655) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_22(8176655) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_68(8176701) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_68(8176701) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_79(8176712) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_79(8176712) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_19(8176652) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_19(8176652) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_14(8176647) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_14(8176647) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_13(8176646) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_13(8176646) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_26(8176659) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_26(8176659) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_25(8176658) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_25(8176658) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_242(8170886) WEXITSTATUS 0 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_242(8170886) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_5(8176638) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_5(8176638) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_70(8176703) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_70(8176703) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_4(8176637) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_4(8176637) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_67(8176700) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_67(8176700) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_72(8176705) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_72(8176705) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_73(8176706) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_73(8176706) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_74(8176707) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_74(8176707) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_12(8176645) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_12(8176645) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_3(8176636) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_3(8176636) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_2(8176635) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_2(8176635) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_81(8176714) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_81(8176714) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_27(8176660) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_27(8176660) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_31(8176664) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_31(8176664) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_28(8176661) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_28(8176661) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_80(8176713) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_80(8176713) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_75(8176708) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_75(8176708) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_32(8176665) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_32(8176665) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_30(8176663) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_30(8176663) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_61(8176694) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_61(8176694) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_29(8176662) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_29(8176662) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_77(8176710) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_77(8176710) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_1(8176634) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_1(8176634) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_71(8176704) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_71(8176704) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_33(8176666) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_33(8176666) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_58(8176691) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_58(8176691) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_59(8176692) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_59(8176692) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_9(8176642) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_9(8176642) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_8(8176641) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_8(8176641) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_83(8176716) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_83(8176716) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_82(8176715) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_82(8176715) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_93(8176726) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_93(8176726) done Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_92(8176725) WEXITSTATUS 1 Nov 05 12:23:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_92(8176725) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_62(8176695) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_62(8176695) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_63(8176696) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_63(8176696) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_90(8176723) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_90(8176723) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_78(8176711) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_78(8176711) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_36(8176669) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_36(8176669) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_89(8176722) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_89(8176722) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_66(8176699) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_66(8176699) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_84(8176717) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_84(8176717) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_96(8175668) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_96(8175668) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_55(8176688) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_55(8176688) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_34(8176667) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_34(8176667) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_60(8176693) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_60(8176693) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_88(8176721) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_88(8176721) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_40(8176673) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_40(8176673) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_87(8176720) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_87(8176720) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_39(8176672) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_39(8176672) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170957 WEXITSTATUS 0 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170957 done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_38(8176671) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_38(8176671) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_64(8176697) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_64(8176697) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_76(8176709) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_76(8176709) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_54(8176687) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_54(8176687) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_7(8176640) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_7(8176640) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_85(8176718) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_85(8176718) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_86(8176719) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_86(8176719) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_6(8176639) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_6(8176639) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_69(8176702) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_69(8176702) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_37(8176670) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_37(8176670) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_65(8176698) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_65(8176698) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_35(8176668) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_35(8176668) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_57(8176690) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_57(8176690) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_56(8176689) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_56(8176689) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_91(8176724) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_91(8176724) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_327(8171046) WEXITSTATUS 0 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_327(8171046) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_53(8176686) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_53(8176686) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_94(8176727) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_94(8176727) done Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_95(8176728) WEXITSTATUS 1 Nov 05 12:23:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_95(8176728) done Nov 05 12:23:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2364(8165557) WEXITSTATUS 0 Nov 05 12:23:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2364(8165557) done Nov 05 12:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_43(8176676) WEXITSTATUS 1 Nov 05 12:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_43(8176676) done Nov 05 12:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_42(8176675) WEXITSTATUS 1 Nov 05 12:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_42(8176675) done Nov 05 12:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_41(8176674) WEXITSTATUS 1 Nov 05 12:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_41(8176674) done Nov 05 12:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_44(8176677) WEXITSTATUS 1 Nov 05 12:23:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175668_44(8176677) done Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173004 NodeList=r805u23n02 #CPUs=16 Partition=day Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175605 NodeList=r816u03n09 #CPUs=8 Partition=scavenge Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175606 NodeList=r816u15n09 #CPUs=8 Partition=scavenge Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175607 NodeList=r816u29n03 #CPUs=8 Partition=scavenge Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175608 NodeList=r816u29n03 #CPUs=8 Partition=scavenge Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175609 NodeList=r816u29n03 #CPUs=8 Partition=scavenge Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_362(8171081) WEXITSTATUS 0 Nov 05 12:23:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_362(8171081) done Nov 05 12:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8044(8172425) WEXITSTATUS 1 Nov 05 12:23:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8044(8172425) done Nov 05 12:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175581 WEXITSTATUS 0 Nov 05 12:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175581 done Nov 05 12:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175580 WEXITSTATUS 0 Nov 05 12:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175580 done Nov 05 12:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175579 WEXITSTATUS 0 Nov 05 12:23:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175579 done Nov 05 12:23:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:23:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_655(8171628) WEXITSTATUS 0 Nov 05 12:23:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_655(8171628) done Nov 05 12:23:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175578 WEXITSTATUS 0 Nov 05 12:23:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175578 done Nov 05 12:23:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175582 WEXITSTATUS 0 Nov 05 12:23:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175582 done Nov 05 12:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_797(8171864) WEXITSTATUS 0 Nov 05 12:24:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_797(8171864) done Nov 05 12:24:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_80(8170620) WEXITSTATUS 0 Nov 05 12:24:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_80(8170620) done Nov 05 12:24:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175583 WEXITSTATUS 0 Nov 05 12:24:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175583 done Nov 05 12:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_329(8171048) WEXITSTATUS 0 Nov 05 12:24:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_329(8171048) done Nov 05 12:24:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_238(8170882) WEXITSTATUS 0 Nov 05 12:24:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_238(8170882) done Nov 05 12:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_708(8171763) WEXITSTATUS 0 Nov 05 12:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_708(8171763) done Nov 05 12:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_498(8171294) WEXITSTATUS 0 Nov 05 12:24:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_498(8171294) done Nov 05 12:24:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_160(8170744) WEXITSTATUS 0 Nov 05 12:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_160(8170744) done Nov 05 12:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_350(8171069) WEXITSTATUS 0 Nov 05 12:24:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_350(8171069) done Nov 05 12:24:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8172685 Nov 05 12:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_265(8170909) WEXITSTATUS 0 Nov 05 12:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_265(8170909) done Nov 05 12:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_497(8171293) WEXITSTATUS 0 Nov 05 12:24:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_497(8171293) done Nov 05 12:24:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_803(8171870) WEXITSTATUS 0 Nov 05 12:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_803(8171870) done Nov 05 12:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_696(8171669) WEXITSTATUS 0 Nov 05 12:24:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_696(8171669) done Nov 05 12:24:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_491(8171287) WEXITSTATUS 0 Nov 05 12:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_491(8171287) done Nov 05 12:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_363(8171082) WEXITSTATUS 0 Nov 05 12:24:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_363(8171082) done Nov 05 12:24:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_713(8171768) WEXITSTATUS 0 Nov 05 12:24:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_713(8171768) done Nov 05 12:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_618(8171591) WEXITSTATUS 0 Nov 05 12:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_618(8171591) done Nov 05 12:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175576 WEXITSTATUS 0 Nov 05 12:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175576 done Nov 05 12:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175577 WEXITSTATUS 0 Nov 05 12:24:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175577 done Nov 05 12:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_348(8171067) WEXITSTATUS 0 Nov 05 12:24:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_348(8171067) done Nov 05 12:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_782(8171848) WEXITSTATUS 0 Nov 05 12:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_782(8171848) done Nov 05 12:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_243(8170887) WEXITSTATUS 0 Nov 05 12:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_243(8170887) done Nov 05 12:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_249(8170893) WEXITSTATUS 0 Nov 05 12:24:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_249(8170893) done Nov 05 12:24:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_215(8170859) WEXITSTATUS 0 Nov 05 12:24:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_215(8170859) done Nov 05 12:24:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_277(8170921) WEXITSTATUS 0 Nov 05 12:24:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_277(8170921) done Nov 05 12:24:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_687(8171660) WEXITSTATUS 0 Nov 05 12:24:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_687(8171660) done Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4038(8176633) WEXITSTATUS 0 Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4038(8176633) done Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_279(8170923) WEXITSTATUS 0 Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_279(8170923) done Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_271(8170915) WEXITSTATUS 0 Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_271(8170915) done Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_489(8171285) WEXITSTATUS 0 Nov 05 12:24:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_489(8171285) done Nov 05 12:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_264(8170908) WEXITSTATUS 0 Nov 05 12:24:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_264(8170908) done Nov 05 12:24:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_203(8170847) WEXITSTATUS 0 Nov 05 12:24:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_203(8170847) done Nov 05 12:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_654(8171627) WEXITSTATUS 0 Nov 05 12:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_654(8171627) done Nov 05 12:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_717(8171772) WEXITSTATUS 0 Nov 05 12:24:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_717(8171772) done Nov 05 12:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_714(8171769) WEXITSTATUS 0 Nov 05 12:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_714(8171769) done Nov 05 12:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_712(8171767) WEXITSTATUS 0 Nov 05 12:24:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_712(8171767) done Nov 05 12:24:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_275(8170919) WEXITSTATUS 0 Nov 05 12:24:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_275(8170919) done Nov 05 12:24:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176731 InitPrio=31343 usec=4791 Nov 05 12:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7443(8166258) WEXITSTATUS 1 Nov 05 12:24:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7443(8166258) done Nov 05 12:24:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7970(8170951) WEXITSTATUS 1 Nov 05 12:24:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7970(8170951) done Nov 05 12:24:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_487(8171283) WEXITSTATUS 0 Nov 05 12:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_487(8171283) done Nov 05 12:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_715(8171770) WEXITSTATUS 0 Nov 05 12:24:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_715(8171770) done Nov 05 12:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7612(8167057) WEXITSTATUS 1 Nov 05 12:24:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7612(8167057) done Nov 05 12:24:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8166304 Nov 05 12:24:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_483(8171279) WEXITSTATUS 0 Nov 05 12:24:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_483(8171279) done Nov 05 12:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_817(8171984) WEXITSTATUS 0 Nov 05 12:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_817(8171984) done Nov 05 12:24:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176732 InitPrio=31343 usec=4736 Nov 05 12:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_224(8170868) WEXITSTATUS 0 Nov 05 12:24:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_224(8170868) done Nov 05 12:24:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_222(8170866) WEXITSTATUS 0 Nov 05 12:24:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_222(8170866) done Nov 05 12:24:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_276(8170920) WEXITSTATUS 0 Nov 05 12:24:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_276(8170920) done Nov 05 12:24:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_244(8170888) WEXITSTATUS 0 Nov 05 12:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_244(8170888) done Nov 05 12:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_20(8175449) WEXITSTATUS 0 Nov 05 12:24:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_20(8175449) done Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_261(8170905) WEXITSTATUS 0 Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_261(8170905) done Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_262(8170906) WEXITSTATUS 0 Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_262(8170906) done Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_592(8171497) WEXITSTATUS 0 Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_592(8171497) done Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_158(8170742) WEXITSTATUS 0 Nov 05 12:24:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_158(8170742) done Nov 05 12:24:48 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8176733 NodeList=(null) usec=23794 Nov 05 12:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3772(8143367) WEXITSTATUS 1 Nov 05 12:24:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3772(8143367) done Nov 05 12:24:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_208(8170852) WEXITSTATUS 0 Nov 05 12:24:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_208(8170852) done Nov 05 12:24:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176733 NodeList=r806u25n01 #CPUs=4 Partition=devel Nov 05 12:24:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_263(8170907) WEXITSTATUS 0 Nov 05 12:24:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_263(8170907) done Nov 05 12:24:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_493(8171289) WEXITSTATUS 0 Nov 05 12:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_493(8171289) done Nov 05 12:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_197(8170781) WEXITSTATUS 0 Nov 05 12:24:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_197(8170781) done Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176731 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176732 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173005 NodeList=r816u09n01 #CPUs=16 Partition=day Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175610 NodeList=r816u09n09 #CPUs=8 Partition=scavenge Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144162 NodeList=r914u13n[03-04] #CPUs=72 Partition=scavenge Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144165 NodeList=r801u27n03,r802u25n04 #CPUs=72 Partition=scavenge Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144167 NodeList=r908u22n02,r916u01n02 #CPUs=72 Partition=scavenge Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_289(8170933) WEXITSTATUS 0 Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_289(8170933) done Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_287(8170931) WEXITSTATUS 0 Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_287(8170931) done Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_259(8170903) WEXITSTATUS 0 Nov 05 12:24:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_259(8170903) done Nov 05 12:24:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_486(8171282) WEXITSTATUS 0 Nov 05 12:24:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_486(8171282) done Nov 05 12:24:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:24:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175584 WEXITSTATUS 0 Nov 05 12:24:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175584 done Nov 05 12:24:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175585 WEXITSTATUS 0 Nov 05 12:24:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175585 done Nov 05 12:25:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_622(8171595) WEXITSTATUS 0 Nov 05 12:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_622(8171595) done Nov 05 12:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_499(8171295) WEXITSTATUS 0 Nov 05 12:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_499(8171295) done Nov 05 12:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_801(8171868) WEXITSTATUS 0 Nov 05 12:25:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_801(8171868) done Nov 05 12:25:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8051(8172489) WEXITSTATUS 1 Nov 05 12:25:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8051(8172489) done Nov 05 12:25:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_231(8170875) WEXITSTATUS 0 Nov 05 12:25:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_231(8170875) done Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175669_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175765_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175861_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8175957_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176053_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176149_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176245_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176341_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_1 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_3 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_4 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_2 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_11 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_88 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_8 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_5 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_10 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176437_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_12 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_9 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_6 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_15 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_17 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_22 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_18 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_24 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_16 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_21 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_27 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_19 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_20 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_13 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_14 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_23 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_28 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_25 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_30 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_32 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_29 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_31 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_37 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_33 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_26 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_36 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_34 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_35 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_38 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_39 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_7 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_40 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_41 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_42 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_43 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_44 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_45 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_47 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_48 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_50 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_56 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_49 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_46 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_54 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_51 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_52 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_53 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_55 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_57 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_66 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_62 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_59 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_60 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_65 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_67 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_64 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_69 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_58 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_71 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_76 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_72 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_73 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_79 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_75 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_77 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_78 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_63 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_68 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_61 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_74 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_83 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_84 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_85 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_82 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_87 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_80 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_89 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_86 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_91 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_92 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_93 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_90 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_70 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_96 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_81 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_95 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_94 uid 23340 Nov 05 12:25:07 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176533_88 uid 23340 Nov 05 12:25:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3382(8135185) Nov 05 12:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_204(8170848) WEXITSTATUS 0 Nov 05 12:25:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_204(8170848) done Nov 05 12:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_199(8170783) WEXITSTATUS 0 Nov 05 12:25:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_199(8170783) done Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:25:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_236(8170880) WEXITSTATUS 0 Nov 05 12:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_236(8170880) done Nov 05 12:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_237(8170881) WEXITSTATUS 0 Nov 05 12:25:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_237(8170881) done Nov 05 12:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2379(8165586) WEXITSTATUS 0 Nov 05 12:25:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2379(8165586) done Nov 05 12:25:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_328(8171047) WEXITSTATUS 0 Nov 05 12:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_328(8171047) done Nov 05 12:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_285(8170929) WEXITSTATUS 0 Nov 05 12:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_285(8170929) done Nov 05 12:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_284(8170928) WEXITSTATUS 0 Nov 05 12:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_284(8170928) done Nov 05 12:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_482(8171278) WEXITSTATUS 0 Nov 05 12:25:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_482(8171278) done Nov 05 12:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_485(8171281) WEXITSTATUS 0 Nov 05 12:25:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_485(8171281) done Nov 05 12:25:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_815(8171982) WEXITSTATUS 0 Nov 05 12:25:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_815(8171982) done Nov 05 12:25:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_214(8170858) WEXITSTATUS 0 Nov 05 12:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_214(8170858) done Nov 05 12:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_205(8170849) WEXITSTATUS 0 Nov 05 12:25:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_205(8170849) done Nov 05 12:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_593(8171498) WEXITSTATUS 0 Nov 05 12:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_593(8171498) done Nov 05 12:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2396(8165608) WEXITSTATUS 0 Nov 05 12:25:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2396(8165608) done Nov 05 12:25:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_278(8170922) WEXITSTATUS 0 Nov 05 12:25:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_278(8170922) done Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175638 WEXITSTATUS 0 Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175638 done Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n04 Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n03 Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u11n01 Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n04 Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n03 Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n02 Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n01 Nov 05 12:25:24 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u11n02 Nov 05 12:25:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_226(8170870) WEXITSTATUS 0 Nov 05 12:25:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_226(8170870) done Nov 05 12:25:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_228(8170872) WEXITSTATUS 0 Nov 05 12:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_228(8170872) done Nov 05 12:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_217(8170861) WEXITSTATUS 0 Nov 05 12:25:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_217(8170861) done Nov 05 12:25:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_823(8171997) WEXITSTATUS 0 Nov 05 12:25:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_823(8171997) done Nov 05 12:25:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_223(8170867) WEXITSTATUS 0 Nov 05 12:25:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_223(8170867) done Nov 05 12:25:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2410(8165641) WEXITSTATUS 0 Nov 05 12:25:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2410(8165641) done Nov 05 12:25:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:37 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176734 InitPrio=74333 usec=7217 Nov 05 12:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_213(8170857) WEXITSTATUS 0 Nov 05 12:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_213(8170857) done Nov 05 12:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_528(8171433) WEXITSTATUS 0 Nov 05 12:25:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_528(8171433) done Nov 05 12:25:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176734 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 12:25:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2746(8153253) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2772(8153279) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2773(8153280) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2782(8153289) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2798(8153305) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2802(8153309) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2810(8153317) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2815(8153322) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2819(8153326) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2825(8153332) Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_619(8171592) WEXITSTATUS 0 Nov 05 12:25:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_619(8171592) done Nov 05 12:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_211(8170855) WEXITSTATUS 0 Nov 05 12:25:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_211(8170855) done Nov 05 12:25:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_539(8171444) WEXITSTATUS 0 Nov 05 12:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_539(8171444) done Nov 05 12:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_551(8171456) WEXITSTATUS 0 Nov 05 12:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_551(8171456) done Nov 05 12:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3649(8137586) WEXITSTATUS 1 Nov 05 12:25:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3649(8137586) done Nov 05 12:25:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_270(8170914) WEXITSTATUS 0 Nov 05 12:25:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_270(8170914) done Nov 05 12:25:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_572(8171477) WEXITSTATUS 0 Nov 05 12:25:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_572(8171477) done Nov 05 12:25:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_662(8171635) WEXITSTATUS 0 Nov 05 12:25:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_662(8171635) done Nov 05 12:25:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_206(8170850) WEXITSTATUS 0 Nov 05 12:25:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_206(8170850) done Nov 05 12:25:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_822(8171996) WEXITSTATUS 0 Nov 05 12:25:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_822(8171996) done Nov 05 12:25:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_490(8171286) WEXITSTATUS 0 Nov 05 12:25:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_490(8171286) done Nov 05 12:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_260(8170904) WEXITSTATUS 0 Nov 05 12:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_260(8170904) done Nov 05 12:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_824(8171998) WEXITSTATUS 0 Nov 05 12:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_824(8171998) done Nov 05 12:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_826(8172000) WEXITSTATUS 0 Nov 05 12:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_826(8172000) done Nov 05 12:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_825(8171999) WEXITSTATUS 0 Nov 05 12:25:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_825(8171999) done Nov 05 12:25:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_303(8171022) WEXITSTATUS 0 Nov 05 12:25:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_303(8171022) done Nov 05 12:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_187(8170771) WEXITSTATUS 0 Nov 05 12:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_187(8170771) done Nov 05 12:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_570(8171475) WEXITSTATUS 0 Nov 05 12:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_570(8171475) done Nov 05 12:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_245(8170889) WEXITSTATUS 0 Nov 05 12:25:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_245(8170889) done Nov 05 12:25:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_690(8171663) WEXITSTATUS 0 Nov 05 12:25:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_690(8171663) done Nov 05 12:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175586 WEXITSTATUS 0 Nov 05 12:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175586 done Nov 05 12:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175587 WEXITSTATUS 0 Nov 05 12:25:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175587 done Nov 05 12:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_210(8170854) WEXITSTATUS 0 Nov 05 12:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_210(8170854) done Nov 05 12:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_783(8171849) WEXITSTATUS 0 Nov 05 12:25:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_783(8171849) done Nov 05 12:25:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:25:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7916(8170359) WEXITSTATUS 1 Nov 05 12:25:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7916(8170359) done Nov 05 12:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_820(8171994) WEXITSTATUS 0 Nov 05 12:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_820(8171994) done Nov 05 12:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_661(8171634) WEXITSTATUS 0 Nov 05 12:26:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_661(8171634) done Nov 05 12:26:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_496(8171292) WEXITSTATUS 0 Nov 05 12:26:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_496(8171292) done Nov 05 12:26:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_495(8171291) WEXITSTATUS 0 Nov 05 12:26:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_495(8171291) done Nov 05 12:26:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_492(8171288) WEXITSTATUS 0 Nov 05 12:26:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_492(8171288) done Nov 05 12:26:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_209(8170853) WEXITSTATUS 0 Nov 05 12:26:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_209(8170853) done Nov 05 12:26:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_455(8171251) WEXITSTATUS 0 Nov 05 12:26:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_455(8171251) done Nov 05 12:26:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3383(8135186) Nov 05 12:26:08 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3388(8135191) Nov 05 12:26:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7535(8166667) WEXITSTATUS 1 Nov 05 12:26:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7535(8166667) done Nov 05 12:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135449 WEXITSTATUS 0 Nov 05 12:26:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135449 done Nov 05 12:26:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4039(8176735) in gpu on r4513u20n01 Nov 05 12:26:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4040(8176736) in gpu on r907u32n01 Nov 05 12:26:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 12:26:11 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2989(8176737) in day on r815u29n01 Nov 05 12:26:12 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u28n01 Nov 05 12:26:12 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u28n02 Nov 05 12:26:12 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u28n04 Nov 05 12:26:12 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u24n04 Nov 05 12:26:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_382(8171101) WEXITSTATUS 0 Nov 05 12:26:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_382(8171101) done Nov 05 12:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_324(8171043) WEXITSTATUS 0 Nov 05 12:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_324(8171043) done Nov 05 12:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_246(8170890) WEXITSTATUS 0 Nov 05 12:26:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_246(8170890) done Nov 05 12:26:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_250(8170894) WEXITSTATUS 0 Nov 05 12:26:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_250(8170894) done Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4232(8152485) in scavenge on r907u01n03 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3421(8135251) in scavenge on r908u33n03 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3731(8141478) in scavenge on r802u14n02 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3819(8144433) in scavenge on r903u07n01 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3821(8144435) in scavenge on r816u03n12 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_829(8122752) in scavenge on r816u23n11 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2983(8128776) in scavenge on r815u09n10 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2977(8128770) in scavenge on r815u09n11 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4007(8146372) in scavenge on r4513u29n01 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3815(8144429) in scavenge on r909u10n03 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2673(8128025) in scavenge on r909u29n04 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3986(8146135) in scavenge on r912u36n04 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_464(7572632) in scavenge on r815u23n09 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_843(7573012) in scavenge on r805u23n02 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_204(8114470) in scavenge on r913u05n03 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1019(7573188) in scavenge on r815u23n02 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_474(7572642) in scavenge on r815u35n03 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_956(7573125) in scavenge on r816u23n06 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_745(7571710) in scavenge on r816u29n02 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_36(8114302) in scavenge on r816u09n05 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8115470_278(8115748) in scavenge on r816u09n05 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_964(7573133) in scavenge on r816u09n06 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_952(8115221) in scavenge on r816u09n07 Nov 05 12:26:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_880(7573049) in scavenge on r816u09n07 Nov 05 12:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_229(8170873) WEXITSTATUS 0 Nov 05 12:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_229(8170873) done Nov 05 12:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_207(8170851) WEXITSTATUS 0 Nov 05 12:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_207(8170851) done Nov 05 12:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_829(8172003) WEXITSTATUS 0 Nov 05 12:26:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_829(8172003) done Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2634(8166297) in scavenge on r816u23n07 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2405(8165636) in scavenge on r816u23n07 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2429(8165678) in scavenge on r816u03n12 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2441(8165715) in scavenge on r816u09n09 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2450(8165733) in scavenge on r816u09n09 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2468(8165779) in scavenge on r915u09n01 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_494(8171290) WEXITSTATUS 0 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_494(8171290) done Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4040(8176736) WEXITSTATUS 0 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4040(8176736) done Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_267(8170911) WEXITSTATUS 0 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_267(8170911) done Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_248(8170892) WEXITSTATUS 0 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_248(8170892) done Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_225(8170869) WEXITSTATUS 0 Nov 05 12:26:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_225(8170869) done Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175591 WEXITSTATUS 0 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175591 done Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175592 WEXITSTATUS 0 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175592 done Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175597 WEXITSTATUS 0 Nov 05 12:26:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175597 done Nov 05 12:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175590 WEXITSTATUS 0 Nov 05 12:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175590 done Nov 05 12:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175604 WEXITSTATUS 0 Nov 05 12:26:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175604 done Nov 05 12:26:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175588 WEXITSTATUS 0 Nov 05 12:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175588 done Nov 05 12:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175589 WEXITSTATUS 0 Nov 05 12:26:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175589 done Nov 05 12:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_38(8175480) WEXITSTATUS 0 Nov 05 12:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_38(8175480) done Nov 05 12:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175596 WEXITSTATUS 0 Nov 05 12:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175596 done Nov 05 12:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_688(8171661) WEXITSTATUS 0 Nov 05 12:26:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_688(8171661) done Nov 05 12:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175595 WEXITSTATUS 0 Nov 05 12:26:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175595 done Nov 05 12:26:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_212(8170856) WEXITSTATUS 0 Nov 05 12:26:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_212(8170856) done Nov 05 12:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175598 WEXITSTATUS 0 Nov 05 12:26:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175598 done Nov 05 12:26:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175594 WEXITSTATUS 0 Nov 05 12:26:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175594 done Nov 05 12:26:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175600 WEXITSTATUS 0 Nov 05 12:26:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175600 done Nov 05 12:26:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175601 WEXITSTATUS 0 Nov 05 12:26:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175601 done Nov 05 12:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_233(8170877) WEXITSTATUS 0 Nov 05 12:26:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_233(8170877) done Nov 05 12:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175603 WEXITSTATUS 0 Nov 05 12:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175603 done Nov 05 12:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175599 WEXITSTATUS 0 Nov 05 12:26:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175599 done Nov 05 12:26:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_309(8171028) WEXITSTATUS 0 Nov 05 12:26:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_309(8171028) done Nov 05 12:26:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175602 WEXITSTATUS 0 Nov 05 12:26:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175602 done Nov 05 12:26:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_488(8171284) WEXITSTATUS 0 Nov 05 12:26:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_488(8171284) done Nov 05 12:26:38 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_355(7571320) Nov 05 12:26:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_8061(8172499) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_561(8171466) WEXITSTATUS 0 Nov 05 12:26:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_561(8171466) done Nov 05 12:26:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_8061(8172499) Nov 05 12:26:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7742(8168845) WEXITSTATUS 1 Nov 05 12:26:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7742(8168845) done Nov 05 12:26:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_828(8172002) WEXITSTATUS 0 Nov 05 12:26:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_828(8172002) done Nov 05 12:26:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_378(8171097) WEXITSTATUS 0 Nov 05 12:26:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_378(8171097) done Nov 05 12:26:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2279(8165123) WEXITSTATUS 0 Nov 05 12:26:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2279(8165123) done Nov 05 12:26:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_534(8171439) WEXITSTATUS 0 Nov 05 12:26:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_534(8171439) done Nov 05 12:26:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_304(8171023) WEXITSTATUS 0 Nov 05 12:26:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_304(8171023) done Nov 05 12:26:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_256(8170900) WEXITSTATUS 0 Nov 05 12:26:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_256(8170900) done Nov 05 12:26:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_253(8170897) WEXITSTATUS 0 Nov 05 12:26:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_253(8170897) done Nov 05 12:26:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175605 WEXITSTATUS 0 Nov 05 12:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175605 done Nov 05 12:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4039(8176735) WEXITSTATUS 0 Nov 05 12:26:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4039(8176735) done Nov 05 12:26:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173006 NodeList=r815u15n02 #CPUs=16 Partition=day Nov 05 12:26:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173007 NodeList=r816u15n03 #CPUs=16 Partition=day Nov 05 12:26:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175606 WEXITSTATUS 0 Nov 05 12:26:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175606 done Nov 05 12:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_254(8170898) WEXITSTATUS 0 Nov 05 12:26:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_254(8170898) done Nov 05 12:26:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_531(8171436) WEXITSTATUS 0 Nov 05 12:26:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_531(8171436) done Nov 05 12:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_537(8171442) WEXITSTATUS 0 Nov 05 12:27:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_537(8171442) done Nov 05 12:27:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175608 WEXITSTATUS 0 Nov 05 12:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175608 done Nov 05 12:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175607 WEXITSTATUS 0 Nov 05 12:27:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175607 done Nov 05 12:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_388(8171107) WEXITSTATUS 0 Nov 05 12:27:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_388(8171107) done Nov 05 12:27:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175609 WEXITSTATUS 0 Nov 05 12:27:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175609 done Nov 05 12:27:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_532(8171437) WEXITSTATUS 0 Nov 05 12:27:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_532(8171437) done Nov 05 12:27:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174108 WEXITSTATUS 1 Nov 05 12:27:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174108 done Nov 05 12:27:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_392(8171111) WEXITSTATUS 0 Nov 05 12:27:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_392(8171111) done Nov 05 12:27:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_393(8171112) WEXITSTATUS 0 Nov 05 12:27:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_393(8171112) done Nov 05 12:27:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_308(8171027) WEXITSTATUS 0 Nov 05 12:27:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_308(8171027) done Nov 05 12:27:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_863(8172126) WEXITSTATUS 0 Nov 05 12:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_863(8172126) done Nov 05 12:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_196(8170780) WEXITSTATUS 0 Nov 05 12:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_196(8170780) done Nov 05 12:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8064(8172572) WEXITSTATUS 1 Nov 05 12:27:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8064(8172572) done Nov 05 12:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175593 WEXITSTATUS 0 Nov 05 12:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175593 done Nov 05 12:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_563(8171468) WEXITSTATUS 0 Nov 05 12:27:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_563(8171468) done Nov 05 12:27:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_307(8171026) WEXITSTATUS 0 Nov 05 12:27:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_307(8171026) done Nov 05 12:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8055(8172493) WEXITSTATUS 1 Nov 05 12:27:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8055(8172493) done Nov 05 12:27:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_391(8171110) WEXITSTATUS 0 Nov 05 12:27:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_391(8171110) done Nov 05 12:27:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_198(8170782) WEXITSTATUS 0 Nov 05 12:27:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_198(8170782) done Nov 05 12:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_527(8171432) WEXITSTATUS 0 Nov 05 12:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_527(8171432) done Nov 05 12:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_290(8170934) WEXITSTATUS 0 Nov 05 12:27:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_290(8170934) done Nov 05 12:27:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_867(8172130) WEXITSTATUS 0 Nov 05 12:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_867(8172130) done Nov 05 12:27:28 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176738 InitPrio=29089 usec=3885 Nov 05 12:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_451(8171247) WEXITSTATUS 0 Nov 05 12:27:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_451(8171247) done Nov 05 12:27:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_557(8171462) WEXITSTATUS 0 Nov 05 12:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_557(8171462) done Nov 05 12:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_35(8175474) WEXITSTATUS 0 Nov 05 12:27:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_35(8175474) done Nov 05 12:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_542(8171447) WEXITSTATUS 0 Nov 05 12:27:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_542(8171447) done Nov 05 12:27:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7233(8165886) WEXITSTATUS 1 Nov 05 12:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7233(8165886) done Nov 05 12:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_524(8171429) WEXITSTATUS 0 Nov 05 12:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_524(8171429) done Nov 05 12:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_302(8171021) WEXITSTATUS 0 Nov 05 12:27:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_302(8171021) done Nov 05 12:27:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_257(8170901) WEXITSTATUS 0 Nov 05 12:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_257(8170901) done Nov 05 12:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_274(8170918) WEXITSTATUS 0 Nov 05 12:27:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_274(8170918) done Nov 05 12:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_574(8171479) WEXITSTATUS 0 Nov 05 12:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_574(8171479) done Nov 05 12:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_541(8171446) WEXITSTATUS 0 Nov 05 12:27:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_541(8171446) done Nov 05 12:27:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:27:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_865(8172128) WEXITSTATUS 0 Nov 05 12:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_865(8172128) done Nov 05 12:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_538(8171443) WEXITSTATUS 0 Nov 05 12:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_538(8171443) done Nov 05 12:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_536(8171441) WEXITSTATUS 0 Nov 05 12:27:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_536(8171441) done Nov 05 12:27:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_305(8171024) WEXITSTATUS 0 Nov 05 12:27:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_305(8171024) done Nov 05 12:27:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_390(8171109) WEXITSTATUS 0 Nov 05 12:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_390(8171109) done Nov 05 12:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_452(8171248) WEXITSTATUS 0 Nov 05 12:27:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_452(8171248) done Nov 05 12:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_871(8172134) WEXITSTATUS 0 Nov 05 12:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_871(8172134) done Nov 05 12:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_406(8171202) WEXITSTATUS 0 Nov 05 12:27:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_406(8171202) done Nov 05 12:27:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_575(8171480) WEXITSTATUS 0 Nov 05 12:27:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_575(8171480) done Nov 05 12:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4030(8175628) WEXITSTATUS 0 Nov 05 12:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4030(8175628) done Nov 05 12:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_332(8171051) WEXITSTATUS 0 Nov 05 12:27:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_332(8171051) done Nov 05 12:27:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_526(8171431) WEXITSTATUS 0 Nov 05 12:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_526(8171431) done Nov 05 12:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_821(8171995) WEXITSTATUS 0 Nov 05 12:27:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_821(8171995) done Nov 05 12:27:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_458(8171254) WEXITSTATUS 0 Nov 05 12:27:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_458(8171254) done Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176738 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173008 NodeList=r815u23n05 #CPUs=16 Partition=day Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8144168 NodeList=r807u39n01,r914u09n04 #CPUs=68 Partition=scavenge Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176739 InitPrio=29089 usec=3828 Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176738 WEXITSTATUS 0 Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176738 done Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_548(8171453) WEXITSTATUS 0 Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_548(8171453) done Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_433(8171229) WEXITSTATUS 0 Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_433(8171229) done Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_429(8171225) WEXITSTATUS 0 Nov 05 12:27:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_429(8171225) done Nov 05 12:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_255(8170899) WEXITSTATUS 0 Nov 05 12:27:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_255(8170899) done Nov 05 12:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_381(8171100) WEXITSTATUS 0 Nov 05 12:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_381(8171100) done Nov 05 12:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_535(8171440) WEXITSTATUS 0 Nov 05 12:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_535(8171440) done Nov 05 12:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_386(8171105) WEXITSTATUS 0 Nov 05 12:27:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_386(8171105) done Nov 05 12:27:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175610 WEXITSTATUS 0 Nov 05 12:27:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175610 done Nov 05 12:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_430(8171226) WEXITSTATUS 0 Nov 05 12:28:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_430(8171226) done Nov 05 12:28:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_870(8172133) WEXITSTATUS 0 Nov 05 12:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_870(8172133) done Nov 05 12:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_525(8171430) WEXITSTATUS 0 Nov 05 12:28:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_525(8171430) done Nov 05 12:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_306(8171025) WEXITSTATUS 0 Nov 05 12:28:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_306(8171025) done Nov 05 12:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_234(8170878) WEXITSTATUS 0 Nov 05 12:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_234(8170878) done Nov 05 12:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_374(8171093) WEXITSTATUS 0 Nov 05 12:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_374(8171093) done Nov 05 12:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_373(8171092) WEXITSTATUS 0 Nov 05 12:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_373(8171092) done Nov 05 12:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_533(8171438) WEXITSTATUS 0 Nov 05 12:28:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_533(8171438) done Nov 05 12:28:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_235(8170879) WEXITSTATUS 0 Nov 05 12:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_235(8170879) done Nov 05 12:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_377(8171096) WEXITSTATUS 0 Nov 05 12:28:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_377(8171096) done Nov 05 12:28:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_566(8171471) WEXITSTATUS 0 Nov 05 12:28:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_566(8171471) done Nov 05 12:28:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_232(8170876) WEXITSTATUS 0 Nov 05 12:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_232(8170876) done Nov 05 12:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_431(8171227) WEXITSTATUS 0 Nov 05 12:28:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_431(8171227) done Nov 05 12:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_384(8171103) WEXITSTATUS 0 Nov 05 12:28:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_384(8171103) done Nov 05 12:28:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_405(8171201) WEXITSTATUS 0 Nov 05 12:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_405(8171201) done Nov 05 12:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_195(8170779) WEXITSTATUS 0 Nov 05 12:28:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_195(8170779) done Nov 05 12:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_827(8172001) WEXITSTATUS 0 Nov 05 12:28:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_827(8172001) done Nov 05 12:28:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_288(8170932) WEXITSTATUS 0 Nov 05 12:28:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_288(8170932) done Nov 05 12:28:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_515(8171420) WEXITSTATUS 0 Nov 05 12:28:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_515(8171420) done Nov 05 12:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176732 WEXITSTATUS 0 Nov 05 12:28:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176732 done Nov 05 12:28:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_219(8170863) WEXITSTATUS 0 Nov 05 12:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_219(8170863) done Nov 05 12:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_421(8171217) WEXITSTATUS 0 Nov 05 12:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_421(8171217) done Nov 05 12:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_862(8172125) WEXITSTATUS 0 Nov 05 12:28:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_862(8172125) done Nov 05 12:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_568(8171473) WEXITSTATUS 0 Nov 05 12:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_568(8171473) done Nov 05 12:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_325(8171044) WEXITSTATUS 0 Nov 05 12:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_325(8171044) done Nov 05 12:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_408(8171204) WEXITSTATUS 0 Nov 05 12:28:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_408(8171204) done Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176731 WEXITSTATUS 0 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176731 done Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_385(8171104) WEXITSTATUS 0 Nov 05 12:28:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_385(8171104) done Nov 05 12:28:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_22(8175458) WEXITSTATUS 0 Nov 05 12:28:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_22(8175458) done Nov 05 12:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_280(8170924) WEXITSTATUS 0 Nov 05 12:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_280(8170924) done Nov 05 12:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_516(8171421) WEXITSTATUS 0 Nov 05 12:28:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_516(8171421) done Nov 05 12:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_530(8171435) WEXITSTATUS 0 Nov 05 12:28:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_530(8171435) done Nov 05 12:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_545(8171450) WEXITSTATUS 0 Nov 05 12:28:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_545(8171450) done Nov 05 12:28:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_869(8172132) WEXITSTATUS 0 Nov 05 12:28:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_869(8172132) done Nov 05 12:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_547(8171452) WEXITSTATUS 0 Nov 05 12:28:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_547(8171452) done Nov 05 12:28:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7767(8169040) WEXITSTATUS 1 Nov 05 12:28:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7767(8169040) done Nov 05 12:28:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_578(8171483) WEXITSTATUS 0 Nov 05 12:28:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_578(8171483) done Nov 05 12:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_447(8171243) WEXITSTATUS 0 Nov 05 12:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_447(8171243) done Nov 05 12:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_446(8171242) WEXITSTATUS 0 Nov 05 12:28:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_446(8171242) done Nov 05 12:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_450(8171246) WEXITSTATUS 0 Nov 05 12:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_450(8171246) done Nov 05 12:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_281(8170925) WEXITSTATUS 0 Nov 05 12:28:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_281(8170925) done Nov 05 12:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_453(8171249) WEXITSTATUS 0 Nov 05 12:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_453(8171249) done Nov 05 12:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_562(8171467) WEXITSTATUS 0 Nov 05 12:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_562(8171467) done Nov 05 12:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_478(8171274) WEXITSTATUS 0 Nov 05 12:28:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_478(8171274) done Nov 05 12:28:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_301(8171020) WEXITSTATUS 0 Nov 05 12:28:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_301(8171020) done Nov 05 12:28:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_519(8171424) WEXITSTATUS 0 Nov 05 12:28:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_519(8171424) done Nov 05 12:28:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_558(8171463) WEXITSTATUS 0 Nov 05 12:28:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_558(8171463) done Nov 05 12:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_269(8170913) WEXITSTATUS 0 Nov 05 12:28:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_269(8170913) done Nov 05 12:28:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_544(8171449) WEXITSTATUS 0 Nov 05 12:28:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_544(8171449) done Nov 05 12:28:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_864(8172127) WEXITSTATUS 0 Nov 05 12:28:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_864(8172127) done Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_550(8171455) WEXITSTATUS 0 Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_550(8171455) done Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_549(8171454) WEXITSTATUS 0 Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_549(8171454) done Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_394(8171113) WEXITSTATUS 0 Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_394(8171113) done Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_517(8171422) WEXITSTATUS 0 Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_517(8171422) done Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_375(8171094) WEXITSTATUS 0 Nov 05 12:28:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_375(8171094) done Nov 05 12:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_397(8171116) WEXITSTATUS 0 Nov 05 12:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_397(8171116) done Nov 05 12:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_879(8172157) WEXITSTATUS 0 Nov 05 12:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_879(8172157) done Nov 05 12:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_465(8171261) WEXITSTATUS 0 Nov 05 12:28:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_465(8171261) done Nov 05 12:28:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_434(8171230) WEXITSTATUS 0 Nov 05 12:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_434(8171230) done Nov 05 12:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7596(8167025) WEXITSTATUS 1 Nov 05 12:28:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7596(8167025) done Nov 05 12:28:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:28:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4041(8176740) in gpu on r808u14n01 Nov 05 12:28:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4042(8176741) in gpu on r4513u20n01 Nov 05 12:28:56 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4043(8176742) in gpu on r907u32n01 Nov 05 12:28:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_540(8171445) WEXITSTATUS 0 Nov 05 12:28:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_540(8171445) done Nov 05 12:28:58 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8176743 NodeList=(null) usec=402196 Nov 05 12:28:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_322(8171041) WEXITSTATUS 0 Nov 05 12:28:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_322(8171041) done Nov 05 12:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_571(8171476) WEXITSTATUS 0 Nov 05 12:29:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_571(8171476) done Nov 05 12:29:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_454(8171250) WEXITSTATUS 0 Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_454(8171250) done Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_459(8171255) WEXITSTATUS 0 Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_459(8171255) done Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_866(8172129) WEXITSTATUS 0 Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_866(8172129) done Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_395(8171114) WEXITSTATUS 0 Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_395(8171114) done Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_436(8171232) WEXITSTATUS 0 Nov 05 12:29:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_436(8171232) done Nov 05 12:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_300(8171019) WEXITSTATUS 0 Nov 05 12:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_300(8171019) done Nov 05 12:29:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_323(8171042) WEXITSTATUS 0 Nov 05 12:29:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_323(8171042) done Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3816(8144430) in scavenge on r802u30n03 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2988(8128781) in scavenge on r916u01n01 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3961(8146004) in scavenge on r816u03n09 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4021(8146489) in scavenge on r815u23n07 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3967(8146010) in scavenge on r815u23n10 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3875(8144940) in scavenge on r906u28n01 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2997(8128790) in scavenge on r807u11n04 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1427(8123663) in scavenge on r913u09n01 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1120(7573289) in scavenge on r916u01n01 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_9(8114275) in scavenge on r815u15n11 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_263(8114529) in scavenge on r815u23n08 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_762(8115031) in scavenge on r816u23n07 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_436(7572604) in scavenge on r816u23n09 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_749(7572918) in scavenge on r816u03n10 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_903(7573072) in scavenge on r816u09n09 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_932(7573101) in scavenge on r816u15n09 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_203(8114469) in scavenge on r913u09n01 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1106(8115375) in scavenge on r805u02n03 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7569662_1144(7570806) in scavenge on r805u04n03 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_221(7572389) in scavenge on r806u02n01 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_235(7572403) in scavenge on r806u09n01 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_430(8114697) in scavenge on r816u15n04 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_930(7573099) in scavenge on r816u29n03 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1147(8115416) in scavenge on r816u29n03 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_183(8114449) in scavenge on r816u29n03 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_961(7571927) in scavenge on r815u29n04 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_1029(7571995) in scavenge on r815u35n05 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_105(7572272) in scavenge on r815u03n02 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_121(7572288) in scavenge on r815u03n08 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_125(7572292) in scavenge on r815u03n08 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_128(7572295) in scavenge on r815u15n08 Nov 05 12:29:05 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_161(7572329) in scavenge on r815u15n08 Nov 05 12:29:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2990(8176744) in scavenge on r816u23n06 Nov 05 12:29:07 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2991(8176745) in scavenge on r816u23n06 Nov 05 12:29:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_432(8171228) WEXITSTATUS 0 Nov 05 12:29:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_432(8171228) done Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2834(8153656) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2836(8153658) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2840(8153662) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2845(8153667) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2846(8153668) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2847(8153669) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2853(8153675) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2858(8153680) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2879(8153701) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2884(8153706) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2899(8153721) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2908(8153730) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2909(8153731) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2916(8153738) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2918(8153740) Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_462(8171258) WEXITSTATUS 0 Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_462(8171258) done Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_218(8170862) WEXITSTATUS 0 Nov 05 12:29:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_218(8170862) done Nov 05 12:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_379(8171098) WEXITSTATUS 0 Nov 05 12:29:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_379(8171098) done Nov 05 12:29:10 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n04.grace.ycrc.yale.edu" Nov 05 12:29:10 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n04.grace.ycrc.yale.edu" Nov 05 12:29:10 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u11n02.grace.ycrc.yale.edu" Nov 05 12:29:10 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u11n02.grace.ycrc.yale.edu" Nov 05 12:29:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_880(8172158) WEXITSTATUS 0 Nov 05 12:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_880(8172158) done Nov 05 12:29:11 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n04.grace.ycrc.yale.edu" Nov 05 12:29:11 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n04.grace.ycrc.yale.edu" Nov 05 12:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_444(8171240) WEXITSTATUS 0 Nov 05 12:29:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_444(8171240) done Nov 05 12:29:11 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n03.grace.ycrc.yale.edu" Nov 05 12:29:11 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n03.grace.ycrc.yale.edu" Nov 05 12:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_445(8171241) WEXITSTATUS 0 Nov 05 12:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_445(8171241) done Nov 05 12:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_443(8171239) WEXITSTATUS 0 Nov 05 12:29:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_443(8171239) done Nov 05 12:29:12 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n03.grace.ycrc.yale.edu" Nov 05 12:29:12 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n03.grace.ycrc.yale.edu" Nov 05 12:29:12 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n02.grace.ycrc.yale.edu" Nov 05 12:29:12 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n02.grace.ycrc.yale.edu" Nov 05 12:29:12 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u11n01.grace.ycrc.yale.edu" Nov 05 12:29:12 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u11n01.grace.ycrc.yale.edu" Nov 05 12:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_424(8171220) WEXITSTATUS 0 Nov 05 12:29:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_424(8171220) done Nov 05 12:29:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:14 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n01.grace.ycrc.yale.edu" Nov 05 12:29:14 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n01.grace.ycrc.yale.edu" Nov 05 12:29:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_460(8171256) WEXITSTATUS 0 Nov 05 12:29:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_460(8171256) done Nov 05 12:29:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:29:15 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176746 InitPrio=17484 usec=5197 Nov 05 12:29:15 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:29:16 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:29:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:17 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:29:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3932(8145656) WEXITSTATUS 1 Nov 05 12:29:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3932(8145656) done Nov 05 12:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_435(8171231) WEXITSTATUS 0 Nov 05 12:29:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_435(8171231) done Nov 05 12:29:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_600(8171573) WEXITSTATUS 0 Nov 05 12:29:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_600(8171573) done Nov 05 12:29:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_601(8171574) WEXITSTATUS 0 Nov 05 12:29:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_601(8171574) done Nov 05 12:29:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_884(8172162) WEXITSTATUS 0 Nov 05 12:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_884(8172162) done Nov 05 12:29:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176747 InitPrio=39776 usec=5519 Nov 05 12:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_523(8171428) WEXITSTATUS 0 Nov 05 12:29:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_523(8171428) done Nov 05 12:29:23 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u24n04.grace.ycrc.yale.edu" Nov 05 12:29:23 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u24n04.grace.ycrc.yale.edu" Nov 05 12:29:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7664(8167616) WEXITSTATUS 1 Nov 05 12:29:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7664(8167616) done Nov 05 12:29:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7638(8167093) WEXITSTATUS 1 Nov 05 12:29:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7638(8167093) done Nov 05 12:29:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_440(8171236) WEXITSTATUS 0 Nov 05 12:29:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_440(8171236) done Nov 05 12:29:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_437(8171233) WEXITSTATUS 0 Nov 05 12:29:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_437(8171233) done Nov 05 12:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_438(8171234) WEXITSTATUS 0 Nov 05 12:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_438(8171234) done Nov 05 12:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_380(8171099) WEXITSTATUS 0 Nov 05 12:29:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_380(8171099) done Nov 05 12:29:29 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u28n01.grace.ycrc.yale.edu" Nov 05 12:29:29 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u28n01.grace.ycrc.yale.edu" Nov 05 12:29:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2266(8165086) WEXITSTATUS 0 Nov 05 12:29:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2266(8165086) done Nov 05 12:29:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_376(8171095) WEXITSTATUS 0 Nov 05 12:29:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_376(8171095) done Nov 05 12:29:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_554(8171459) WEXITSTATUS 0 Nov 05 12:29:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_554(8171459) done Nov 05 12:29:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_441(8171237) WEXITSTATUS 0 Nov 05 12:29:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_441(8171237) done Nov 05 12:29:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_410(8171206) WEXITSTATUS 0 Nov 05 12:29:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_410(8171206) done Nov 05 12:29:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_404(8171200) WEXITSTATUS 0 Nov 05 12:29:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_404(8171200) done Nov 05 12:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_399(8171118) WEXITSTATUS 0 Nov 05 12:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_399(8171118) done Nov 05 12:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_521(8171426) WEXITSTATUS 0 Nov 05 12:29:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_521(8171426) done Nov 05 12:29:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4042(8176741) WEXITSTATUS 0 Nov 05 12:29:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4042(8176741) done Nov 05 12:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4041(8176740) WEXITSTATUS 0 Nov 05 12:29:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4041(8176740) done Nov 05 12:29:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_456(8171252) WEXITSTATUS 0 Nov 05 12:29:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_456(8171252) done Nov 05 12:29:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5094(8158418) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:29:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5095(8158419) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:29:40 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u28n04.grace.ycrc.yale.edu" Nov 05 12:29:40 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u28n04.grace.ycrc.yale.edu" Nov 05 12:29:40 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r905u28n02.grace.ycrc.yale.edu" Nov 05 12:29:40 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r905u28n02.grace.ycrc.yale.edu" Nov 05 12:29:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_194(8170778) WEXITSTATUS 0 Nov 05 12:29:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_194(8170778) done Nov 05 12:29:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5094(8158418) Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5095(8158419) Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_461(8171257) WEXITSTATUS 0 Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_461(8171257) done Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176743 NodeList=r909u29n02 #CPUs=30 Partition=day Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_573(8171478) WEXITSTATUS 0 Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_573(8171478) done Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_425(8171221) WEXITSTATUS 0 Nov 05 12:29:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_425(8171221) done Nov 05 12:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_439(8171235) WEXITSTATUS 0 Nov 05 12:29:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_439(8171235) done Nov 05 12:29:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_473(8171269) WEXITSTATUS 0 Nov 05 12:29:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_473(8171269) done Nov 05 12:29:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_282(8170926) WEXITSTATUS 0 Nov 05 12:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_282(8170926) done Nov 05 12:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_283(8170927) WEXITSTATUS 0 Nov 05 12:29:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_283(8170927) done Nov 05 12:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2458(8165748) WEXITSTATUS 0 Nov 05 12:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2458(8165748) done Nov 05 12:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_411(8171207) WEXITSTATUS 0 Nov 05 12:29:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_411(8171207) done Nov 05 12:29:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8094(8172735) WEXITSTATUS 1 Nov 05 12:29:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8094(8172735) done Nov 05 12:29:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_983(8172347) WEXITSTATUS 0 Nov 05 12:29:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_983(8172347) done Nov 05 12:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_286(8170930) WEXITSTATUS 0 Nov 05 12:29:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_286(8170930) done Nov 05 12:29:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_470(8171266) WEXITSTATUS 0 Nov 05 12:29:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_470(8171266) done Nov 05 12:29:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:29:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176747 NodeList=r815u23n01 #CPUs=4 Partition=day Nov 05 12:29:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176739 NodeList=r816u09n07 #CPUs=16 Partition=day Nov 05 12:29:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_33(7572200) WEXITSTATUS 0 Nov 05 12:29:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_33(7572200) done Nov 05 12:29:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_407(8171203) WEXITSTATUS 0 Nov 05 12:29:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_407(8171203) done Nov 05 12:29:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_569(8171474) WEXITSTATUS 0 Nov 05 12:29:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_569(8171474) done Nov 05 12:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_576(8171481) WEXITSTATUS 0 Nov 05 12:29:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_576(8171481) done Nov 05 12:29:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_565(8171470) WEXITSTATUS 0 Nov 05 12:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_565(8171470) done Nov 05 12:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_529(8171434) WEXITSTATUS 0 Nov 05 12:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_529(8171434) done Nov 05 12:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_17(8175205) WEXITSTATUS 0 Nov 05 12:30:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_17(8175205) done Nov 05 12:30:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_513(8171418) WEXITSTATUS 0 Nov 05 12:30:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_513(8171418) done Nov 05 12:30:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_423(8171219) WEXITSTATUS 0 Nov 05 12:30:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_423(8171219) done Nov 05 12:30:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_464(8171260) WEXITSTATUS 0 Nov 05 12:30:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_464(8171260) done Nov 05 12:30:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_985(8172349) WEXITSTATUS 0 Nov 05 12:30:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_985(8172349) done Nov 05 12:30:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_479(8171275) WEXITSTATUS 0 Nov 05 12:30:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_479(8171275) done Nov 05 12:30:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_559(8171464) WEXITSTATUS 0 Nov 05 12:30:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_559(8171464) done Nov 05 12:30:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_387(8171106) WEXITSTATUS 0 Nov 05 12:30:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_387(8171106) done Nov 05 12:30:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_868(8172131) WEXITSTATUS 0 Nov 05 12:30:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_868(8172131) done Nov 05 12:30:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176747 WEXITSTATUS 0 Nov 05 12:30:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176747 done Nov 05 12:30:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159167 WEXITSTATUS 0 Nov 05 12:30:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8159167 done Nov 05 12:30:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_409(8171205) WEXITSTATUS 0 Nov 05 12:30:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_409(8171205) done Nov 05 12:30:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_518(8171423) WEXITSTATUS 0 Nov 05 12:30:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_518(8171423) done Nov 05 12:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_372(8171091) WEXITSTATUS 0 Nov 05 12:30:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_372(8171091) done Nov 05 12:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_389(8171108) WEXITSTATUS 0 Nov 05 12:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_389(8171108) done Nov 05 12:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_522(8171427) WEXITSTATUS 0 Nov 05 12:30:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_522(8171427) done Nov 05 12:30:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_420(8171216) WEXITSTATUS 0 Nov 05 12:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_420(8171216) done Nov 05 12:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_419(8171215) WEXITSTATUS 0 Nov 05 12:30:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_419(8171215) done Nov 05 12:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_383(8171102) WEXITSTATUS 0 Nov 05 12:30:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_383(8171102) done Nov 05 12:30:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_520(8171425) WEXITSTATUS 0 Nov 05 12:30:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_520(8171425) done Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:30:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:30:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176748 InitPrio=39761 usec=6074 Nov 05 12:30:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_984(8172348) WEXITSTATUS 0 Nov 05 12:30:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_984(8172348) done Nov 05 12:30:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1456(8137385) WEXITSTATUS 0 Nov 05 12:30:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_1456(8137385) done Nov 05 12:30:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6800(8164956) WEXITSTATUS 1 Nov 05 12:30:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6800(8164956) done Nov 05 12:30:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_982(8172346) WEXITSTATUS 0 Nov 05 12:30:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_982(8172346) done Nov 05 12:30:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_471(8171267) WEXITSTATUS 0 Nov 05 12:30:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_471(8171267) done Nov 05 12:30:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:39 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5503(8160340) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128211 WEXITSTATUS 0 Nov 05 12:30:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8128211 done Nov 05 12:30:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5503(8160340) Nov 05 12:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_552(8171457) WEXITSTATUS 0 Nov 05 12:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_552(8171457) done Nov 05 12:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_396(8171115) WEXITSTATUS 0 Nov 05 12:30:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_396(8171115) done Nov 05 12:30:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_428(8171224) WEXITSTATUS 0 Nov 05 12:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_428(8171224) done Nov 05 12:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_514(8171419) WEXITSTATUS 0 Nov 05 12:30:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_514(8171419) done Nov 05 12:30:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_422(8171218) WEXITSTATUS 0 Nov 05 12:30:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_422(8171218) done Nov 05 12:30:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_417(8171213) WEXITSTATUS 0 Nov 05 12:30:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_417(8171213) done Nov 05 12:30:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_883(8172161) WEXITSTATUS 0 Nov 05 12:30:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_883(8172161) done Nov 05 12:30:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_414(8171210) WEXITSTATUS 0 Nov 05 12:30:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_414(8171210) done Nov 05 12:30:55 r912u23n01 slurmctld[163760]: slurmctld: node r804u11n02 returned to service Nov 05 12:30:55 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n04 returned to service Nov 05 12:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_457(8171253) WEXITSTATUS 0 Nov 05 12:30:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_457(8171253) done Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173009 NodeList=r816u03n01 #CPUs=16 Partition=day Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176746 NodeList=r912u09n01 #CPUs=1 Partition=bigmem Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n03 returned to service Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n04 returned to service Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_555(8171460) WEXITSTATUS 0 Nov 05 12:30:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_555(8171460) done Nov 05 12:30:57 r912u23n01 slurmctld[163760]: slurmctld: node r804u11n01 returned to service Nov 05 12:30:57 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n03 returned to service Nov 05 12:30:57 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n02 returned to service Nov 05 12:30:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_442(8171238) WEXITSTATUS 0 Nov 05 12:30:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_442(8171238) done Nov 05 12:30:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7954(8170788) WEXITSTATUS 1 Nov 05 12:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7954(8170788) done Nov 05 12:30:59 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n01 returned to service Nov 05 12:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_418(8171214) WEXITSTATUS 0 Nov 05 12:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_418(8171214) done Nov 05 12:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_977(8172341) WEXITSTATUS 0 Nov 05 12:30:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_977(8172341) done Nov 05 12:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_412(8171208) WEXITSTATUS 0 Nov 05 12:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_412(8171208) done Nov 05 12:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2332(8165348) WEXITSTATUS 0 Nov 05 12:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2332(8165348) done Nov 05 12:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_981(8172345) WEXITSTATUS 0 Nov 05 12:31:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_981(8172345) done Nov 05 12:31:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_472(8171268) WEXITSTATUS 0 Nov 05 12:31:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_472(8171268) done Nov 05 12:31:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:08 r912u23n01 slurmctld[163760]: slurmctld: node r905u24n04 returned to service Nov 05 12:31:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3397(8135213) Nov 05 12:31:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:14 r912u23n01 slurmctld[163760]: slurmctld: node r905u28n01 returned to service Nov 05 12:31:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_577(8171482) WEXITSTATUS 0 Nov 05 12:31:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_577(8171482) done Nov 05 12:31:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 12:31:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 12:31:21 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 12:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_477(8171273) WEXITSTATUS 0 Nov 05 12:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_477(8171273) done Nov 05 12:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2301(8165187) WEXITSTATUS 0 Nov 05 12:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2301(8165187) done Nov 05 12:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_469(8171265) WEXITSTATUS 0 Nov 05 12:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_469(8171265) done Nov 05 12:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_504(8171409) WEXITSTATUS 0 Nov 05 12:31:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_504(8171409) done Nov 05 12:31:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:24 r912u23n01 slurmctld[163760]: slurmctld: node r905u28n04 returned to service Nov 05 12:31:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_556(8171461) WEXITSTATUS 0 Nov 05 12:31:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_556(8171461) done Nov 05 12:31:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_467(8171263) WEXITSTATUS 0 Nov 05 12:31:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_467(8171263) done Nov 05 12:31:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:25 r912u23n01 slurmctld[163760]: slurmctld: node r905u28n02 returned to service Nov 05 12:31:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_546(8171451) WEXITSTATUS 0 Nov 05 12:31:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_546(8171451) done Nov 05 12:31:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_980(8172344) WEXITSTATUS 0 Nov 05 12:31:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_980(8172344) done Nov 05 12:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_505(8171410) WEXITSTATUS 0 Nov 05 12:31:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_505(8171410) done Nov 05 12:31:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_978(8172342) WEXITSTATUS 0 Nov 05 12:31:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_978(8172342) done Nov 05 12:31:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_416(8171212) WEXITSTATUS 0 Nov 05 12:31:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_416(8171212) done Nov 05 12:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_543(8171448) WEXITSTATUS 0 Nov 05 12:31:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_543(8171448) done Nov 05 12:31:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_474(8171270) WEXITSTATUS 0 Nov 05 12:31:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_474(8171270) done Nov 05 12:31:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173012 in day on r815u09n03 Nov 05 12:31:38 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173013 in day on r815u35n05 Nov 05 12:31:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4044(8176749) in gpu on r808u14n01 Nov 05 12:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4045(8176750) in gpu on r4513u20n01 Nov 05 12:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 12:31:39 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2992(8176751) in day on r815u03n06 Nov 05 12:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2344(8165446) WEXITSTATUS 0 Nov 05 12:31:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2344(8165446) done Nov 05 12:31:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154719 WEXITSTATUS 0 Nov 05 12:31:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154719 done Nov 05 12:31:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_553(8171458) WEXITSTATUS 0 Nov 05 12:31:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_553(8171458) done Nov 05 12:31:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_564(8171469) WEXITSTATUS 0 Nov 05 12:31:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_564(8171469) done Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1357(8123542) in scavenge on r908u35n04 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2898(8128582) in scavenge on r802u04n03 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3637(8137183) in scavenge on r802u09n04 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4076(8148135) in scavenge on r912u32n02 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4408(8155446) in scavenge on r913u28n01 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3186(8131992) in scavenge on r805u02n03 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4120(8149235) in scavenge on r805u06n03 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4239(8152906) in scavenge on r805u25n03 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2996(8128789) in scavenge on r805u25n04 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4279(8153942) in scavenge on r805u27n02 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_177(7572345) in scavenge on r815u35n02 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_182(7572350) in scavenge on r816u03n06 Nov 05 12:31:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_186(7572354) in scavenge on r816u09n08 Nov 05 12:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4035(8175653) WEXITSTATUS 0 Nov 05 12:31:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4035(8175653) done Nov 05 12:31:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2466(8165777) in scavenge on r815u15n09 Nov 05 12:31:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2683(8166484) in scavenge on r912u09n01 Nov 05 12:31:51 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2467(8165778) in scavenge on r912u09n01 Nov 05 12:31:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:51 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 12:31:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_401(8171197) WEXITSTATUS 0 Nov 05 12:31:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_401(8171197) done Nov 05 12:31:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:31:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173010 NodeList=r816u15n05 #CPUs=16 Partition=day Nov 05 12:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_427(8171223) WEXITSTATUS 0 Nov 05 12:31:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_427(8171223) done Nov 05 12:31:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_509(8171414) WEXITSTATUS 0 Nov 05 12:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_509(8171414) done Nov 05 12:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_400(8171196) WEXITSTATUS 0 Nov 05 12:32:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_400(8171196) done Nov 05 12:32:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3400(8135218) Nov 05 12:32:09 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8168454 Nov 05 12:32:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_506(8171411) WEXITSTATUS 0 Nov 05 12:32:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_506(8171411) done Nov 05 12:32:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_18(8175239) WEXITSTATUS 0 Nov 05 12:32:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_18(8175239) done Nov 05 12:32:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_468(8171264) WEXITSTATUS 0 Nov 05 12:32:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_468(8171264) done Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_773(8171839) WEXITSTATUS 0 Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_773(8171839) done Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 29 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 29 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 307200 seconds when only 148480 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 307200 seconds when only 148480 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 29 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 29 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 0 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 307200 seconds when only 0 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 29 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 0 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 0 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 29 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 0 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 307200 seconds when only 0 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 0 remained. Nov 05 12:32:18 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 60 seconds when only 0 remained. Nov 05 12:32:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_503(8171408) WEXITSTATUS 0 Nov 05 12:32:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_503(8171408) done Nov 05 12:32:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:32:25 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_979(8172343) WEXITSTATUS 0 Nov 05 12:32:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_979(8172343) done Nov 05 12:32:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_885(8172163) WEXITSTATUS 0 Nov 05 12:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_885(8172163) done Nov 05 12:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_878(8172156) WEXITSTATUS 0 Nov 05 12:32:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_878(8172156) done Nov 05 12:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_398(8171117) WEXITSTATUS 0 Nov 05 12:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_398(8171117) done Nov 05 12:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_402(8171198) WEXITSTATUS 0 Nov 05 12:32:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_402(8171198) done Nov 05 12:32:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_508(8171413) WEXITSTATUS 0 Nov 05 12:32:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_508(8171413) done Nov 05 12:32:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7989(8171304) WEXITSTATUS 1 Nov 05 12:32:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7989(8171304) done Nov 05 12:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8178(8174117) WEXITSTATUS 1 Nov 05 12:32:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8178(8174117) done Nov 05 12:32:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_881(8172159) WEXITSTATUS 0 Nov 05 12:32:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_881(8172159) done Nov 05 12:32:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:32:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_475(8171271) WEXITSTATUS 0 Nov 05 12:32:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_475(8171271) done Nov 05 12:32:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_789(8171856) WEXITSTATUS 0 Nov 05 12:32:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_789(8171856) done Nov 05 12:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2354(8165492) WEXITSTATUS 0 Nov 05 12:32:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2354(8165492) done Nov 05 12:32:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_481(8171277) WEXITSTATUS 0 Nov 05 12:32:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_481(8171277) done Nov 05 12:32:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_804(8115073) WEXITSTATUS 0 Nov 05 12:32:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_804(8115073) done Nov 05 12:32:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:32:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_567(8171472) WEXITSTATUS 0 Nov 05 12:32:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_567(8171472) done Nov 05 12:33:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_413(8171209) WEXITSTATUS 0 Nov 05 12:33:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_413(8171209) done Nov 05 12:33:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_757(8171823) WEXITSTATUS 0 Nov 05 12:33:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_757(8171823) done Nov 05 12:33:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6709(8164814) WEXITSTATUS 1 Nov 05 12:33:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6709(8164814) done Nov 05 12:33:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2955(8153892) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153892: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153892: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153892: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153892: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153892: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153892: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153892: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153892: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2956(8153893) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153893: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153893: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153893: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153893: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153893: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153893: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2963(8153900) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153900: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153893: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153900: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153900: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153900: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153893: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2966(8153903) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153903: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153903: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153903: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153900: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153903: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2974(8153911) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153911: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153911: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153911: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153900: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153900: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153911: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153900: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2976(8153913) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153913: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153913: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153903: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153913: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153913: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2977(8153914) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153903: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153914: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153914: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153903: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153903: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153911: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153914: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153914: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153911: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2986(8153923) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153911: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153911: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8146707_2988(8153925) Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153913: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153913: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153913: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153913: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153914: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153914: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153914: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153914: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153923: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES cpu grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES mem grp_used_tres_run_secs underflow, tried to remove 133120 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES node grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: error: _handle_assoc_tres_run_secs: job 8153925: assoc 5405 TRES billing grp_used_tres_run_secs underflow, tried to remove 26 seconds when only 0 remained. Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_502(8171407) WEXITSTATUS 0 Nov 05 12:33:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_502(8171407) done Nov 05 12:33:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8002(8171676) WEXITSTATUS 1 Nov 05 12:33:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8002(8171676) done Nov 05 12:33:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_463(8171259) WEXITSTATUS 0 Nov 05 12:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_463(8171259) done Nov 05 12:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 12:33:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 12:33:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:19 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 12:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_976(8172340) WEXITSTATUS 0 Nov 05 12:33:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_976(8172340) done Nov 05 12:33:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_476(8171272) WEXITSTATUS 0 Nov 05 12:33:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_476(8171272) done Nov 05 12:33:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_781(8171847) WEXITSTATUS 0 Nov 05 12:33:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_781(8171847) done Nov 05 12:33:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_415(8171211) WEXITSTATUS 0 Nov 05 12:33:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_415(8171211) done Nov 05 12:33:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_466(8171262) WEXITSTATUS 0 Nov 05 12:33:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_466(8171262) done Nov 05 12:33:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_774(8171840) WEXITSTATUS 0 Nov 05 12:33:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_774(8171840) done Nov 05 12:33:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_761(8171827) WEXITSTATUS 0 Nov 05 12:33:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_761(8171827) done Nov 05 12:33:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_507(8171412) WEXITSTATUS 0 Nov 05 12:33:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_507(8171412) done Nov 05 12:33:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_777(8171843) WEXITSTATUS 0 Nov 05 12:33:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_777(8171843) done Nov 05 12:33:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_762(8171828) WEXITSTATUS 0 Nov 05 12:33:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_762(8171828) done Nov 05 12:33:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7100(8165656) WEXITSTATUS 1 Nov 05 12:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7100(8165656) done Nov 05 12:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_560(8171465) WEXITSTATUS 0 Nov 05 12:33:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_560(8171465) done Nov 05 12:33:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_776(8171842) WEXITSTATUS 0 Nov 05 12:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_776(8171842) done Nov 05 12:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_780(8171846) WEXITSTATUS 0 Nov 05 12:33:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_780(8171846) done Nov 05 12:33:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:33:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173011 NodeList=r815u09n07 #CPUs=16 Partition=day Nov 05 12:33:56 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 12:33:56 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 12:33:57 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 12:33:58 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 12:33:59 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 12:33:59 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 12:34:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_756(8171822) WEXITSTATUS 0 Nov 05 12:34:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_756(8171822) done Nov 05 12:34:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_770(8171836) WEXITSTATUS 0 Nov 05 12:34:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_770(8171836) done Nov 05 12:34:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_772(8171838) WEXITSTATUS 0 Nov 05 12:34:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_772(8171838) done Nov 05 12:34:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176752 InitPrio=20704 usec=3318 Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_0(8176753) NodeList=r914u09n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_1(8176754) NodeList=r914u09n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_2(8176755) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_3(8176756) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_4(8176757) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_5(8176758) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_6(8176759) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_7(8176760) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_8(8176761) NodeList=r915u29n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_9(8176762) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_10(8176763) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:34:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_11(8176764) in day on r904u10n03 Nov 05 12:34:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_12(8176765) in day on r806u09n04 Nov 05 12:34:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_13(8176766) in day on r806u11n03 Nov 05 12:34:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_14(8176767) in day on r815u09n01 Nov 05 12:34:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_15(8176768) in day on r815u09n04 Nov 05 12:34:23 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4046(8176769) in gpu on r903u40n01 Nov 05 12:34:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_758(8171824) WEXITSTATUS 0 Nov 05 12:34:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_758(8171824) done Nov 05 12:34:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_756(7571721) WEXITSTATUS 0 Nov 05 12:34:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_756(7571721) done Nov 05 12:34:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_511(8171416) WEXITSTATUS 0 Nov 05 12:34:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_511(8171416) done Nov 05 12:34:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_755(8171821) WEXITSTATUS 0 Nov 05 12:34:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_755(8171821) done Nov 05 12:34:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_403(8171199) WEXITSTATUS 0 Nov 05 12:34:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_403(8171199) done Nov 05 12:34:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4046(8176769) WEXITSTATUS 0 Nov 05 12:34:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4046(8176769) done Nov 05 12:34:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_512(8171417) WEXITSTATUS 0 Nov 05 12:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_512(8171417) done Nov 05 12:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_426(8171222) WEXITSTATUS 0 Nov 05 12:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_426(8171222) done Nov 05 12:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_763(8171829) WEXITSTATUS 0 Nov 05 12:34:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_763(8171829) done Nov 05 12:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_775(8171841) WEXITSTATUS 0 Nov 05 12:34:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_775(8171841) done Nov 05 12:34:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:35 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3827(8144441) in scavenge on r815u15n09 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4009(8146374) in scavenge on r816u35n09 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4456(8155769) in scavenge on r913u13n02 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2588(8127795) in scavenge on r914u05n03 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4368(8155094) in scavenge on r909u10n02 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4360(8154970) in scavenge on r909u29n01 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_757(7572926) in scavenge on r918u05n03 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_970(7573139) in scavenge on r806u02n02 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_236(8114502) in scavenge on r806u04n04 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_509(8114776) in scavenge on r816u03n03 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_1178(8115447) in scavenge on r816u23n06 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_765(8171831) WEXITSTATUS 0 Nov 05 12:34:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_765(8171831) done Nov 05 12:34:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_791(8171858) WEXITSTATUS 0 Nov 05 12:34:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_791(8171858) done Nov 05 12:34:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_510(8171415) WEXITSTATUS 0 Nov 05 12:34:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_510(8171415) done Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7795(8169272) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:34:40 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:34:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7795(8169272) Nov 05 12:34:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_779(8171845) WEXITSTATUS 0 Nov 05 12:34:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_779(8171845) done Nov 05 12:34:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_760(8171826) WEXITSTATUS 0 Nov 05 12:34:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_760(8171826) done Nov 05 12:34:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2372(8165579) WEXITSTATUS 0 Nov 05 12:34:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2372(8165579) done Nov 05 12:34:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8012(8171813) WEXITSTATUS 1 Nov 05 12:34:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8012(8171813) done Nov 05 12:34:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:34:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173014 NodeList=r914u01n04 #CPUs=16 Partition=day Nov 05 12:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_795(8171862) WEXITSTATUS 0 Nov 05 12:34:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_795(8171862) done Nov 05 12:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_794(8171861) WEXITSTATUS 0 Nov 05 12:34:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_794(8171861) done Nov 05 12:34:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176770 InitPrio=24888 usec=7973 Nov 05 12:34:58 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176770 NodeList=r903u01n02 #CPUs=1 Partition=pi_zhu Nov 05 12:34:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:10 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1002(8122969) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:35:10 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1344(8123526) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:35:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_602(8171575) WEXITSTATUS 0 Nov 05 12:35:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_602(8171575) done Nov 05 12:35:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:12 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1344(8123526) Nov 05 12:35:12 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1002(8122969) Nov 05 12:35:13 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_16(8176771) NodeList=r908u39n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:35:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_793(8171860) WEXITSTATUS 0 Nov 05 12:35:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_793(8171860) done Nov 05 12:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_790(8171857) WEXITSTATUS 0 Nov 05 12:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_790(8171857) done Nov 05 12:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7897(8170104) WEXITSTATUS 1 Nov 05 12:35:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7897(8170104) done Nov 05 12:35:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7774(8169047) WEXITSTATUS 1 Nov 05 12:35:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7774(8169047) done Nov 05 12:35:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_778(8171844) WEXITSTATUS 0 Nov 05 12:35:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_778(8171844) done Nov 05 12:35:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_792(8171859) WEXITSTATUS 0 Nov 05 12:35:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_792(8171859) done Nov 05 12:35:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4045(8176750) WEXITSTATUS 0 Nov 05 12:35:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4045(8176750) done Nov 05 12:35:29 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176772 InitPrio=26208 usec=5976 Nov 05 12:35:30 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176772 NodeList=r806u23n04 #CPUs=1 Partition=devel Nov 05 12:35:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_615(8171588) WEXITSTATUS 0 Nov 05 12:35:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_615(8171588) done Nov 05 12:35:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_771(8171837) WEXITSTATUS 0 Nov 05 12:35:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_771(8171837) done Nov 05 12:35:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8176770 uid 19353 Nov 05 12:35:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_796(8171863) WEXITSTATUS 0 Nov 05 12:35:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_796(8171863) done Nov 05 12:35:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_818(8171992) WEXITSTATUS 0 Nov 05 12:35:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_818(8171992) done Nov 05 12:35:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_448(8171244) WEXITSTATUS 0 Nov 05 12:35:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_448(8171244) done Nov 05 12:35:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:35:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8124(8172809) WEXITSTATUS 1 Nov 05 12:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8124(8172809) done Nov 05 12:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_603(8171576) WEXITSTATUS 0 Nov 05 12:36:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_603(8171576) done Nov 05 12:36:04 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1357(8123542) has been requeued to reclaim resources for JobId=8176752_17(8176752) Nov 05 12:36:04 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1395(8123626) has been requeued to reclaim resources for JobId=8176752_17(8176752) Nov 05 12:36:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1395(8123626) Nov 05 12:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_788(8171855) WEXITSTATUS 0 Nov 05 12:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_788(8171855) done Nov 05 12:36:06 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1357(8123542) Nov 05 12:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_759(8171825) WEXITSTATUS 0 Nov 05 12:36:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_759(8171825) done Nov 05 12:36:07 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_17(8176773) NodeList=r908u35n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:36:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_882(8172160) WEXITSTATUS 0 Nov 05 12:36:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_882(8172160) done Nov 05 12:36:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_766(8171832) WEXITSTATUS 0 Nov 05 12:36:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_766(8171832) done Nov 05 12:36:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176774 InitPrio=31809 usec=7079 Nov 05 12:36:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_767(8171833) WEXITSTATUS 0 Nov 05 12:36:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_767(8171833) done Nov 05 12:36:32 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176775 InitPrio=24888 usec=3253 Nov 05 12:36:35 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176775 NodeList=r903u01n02 #CPUs=1 Partition=pi_zhu Nov 05 12:36:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_769(8171835) WEXITSTATUS 0 Nov 05 12:36:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_769(8171835) done Nov 05 12:36:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_764(8171830) WEXITSTATUS 0 Nov 05 12:36:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_764(8171830) done Nov 05 12:36:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:41 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1826(8124706) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:36:41 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2219(8126107) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:36:42 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:36:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1826(8124706) Nov 05 12:36:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2219(8126107) Nov 05 12:36:44 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_18(8176776) NodeList=r908u37n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:36:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176777 InitPrio=39776 usec=5414 Nov 05 12:36:46 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8170375 uid 20659 Nov 05 12:36:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2259(8165079) WEXITSTATUS 0 Nov 05 12:36:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2259(8165079) done Nov 05 12:36:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_768(8171834) WEXITSTATUS 0 Nov 05 12:36:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_768(8171834) done Nov 05 12:36:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8045(8172426) WEXITSTATUS 1 Nov 05 12:36:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8045(8172426) done Nov 05 12:36:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176777 NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 12:36:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176774 NodeList=r806u33n01 #CPUs=1 Partition=gpu_devel Nov 05 12:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_702(8171757) WEXITSTATUS 0 Nov 05 12:36:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_702(8171757) done Nov 05 12:36:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:36:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_703(8171758) WEXITSTATUS 0 Nov 05 12:36:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_703(8171758) done Nov 05 12:37:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_841(8172015) WEXITSTATUS 0 Nov 05 12:37:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_841(8172015) done Nov 05 12:37:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_480(8171276) WEXITSTATUS 0 Nov 05 12:37:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_480(8171276) done Nov 05 12:37:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176777 WEXITSTATUS 0 Nov 05 12:37:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176777 done Nov 05 12:37:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2253(8165057) WEXITSTATUS 0 Nov 05 12:37:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2253(8165057) done Nov 05 12:37:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_19(8176778) in day on r806u06n04 Nov 05 12:37:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_20(8176779) in day on r918u01n03 Nov 05 12:37:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4047(8176780) in gpu on r4513u20n01 Nov 05 12:37:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4048(8176781) in gpu on r903u40n01 Nov 05 12:37:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n02 Nov 05 12:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_940(7571906) WEXITSTATUS 0 Nov 05 12:37:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_940(7571906) done Nov 05 12:37:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_614(8171587) WEXITSTATUS 0 Nov 05 12:37:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_614(8171587) done Nov 05 12:37:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176775 WEXITSTATUS 0 Nov 05 12:37:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176775 done Nov 05 12:37:26 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176782 InitPrio=29089 usec=4963 Nov 05 12:37:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4615(8156194) in scavenge on r802u09n01 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4366(8155092) in scavenge on r807u03n02 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4465(8155782) in scavenge on r909u10n03 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4365(8155091) in scavenge on r913u24n04 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3783(8143686) in scavenge on r914u01n04 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2911(8128595) in scavenge on r914u01n04 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4220(8152275) in scavenge on r806u06n03 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4317(8154200) in scavenge on r806u09n02 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4660(8156322) in scavenge on r806u11n01 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4306(8154072) in scavenge on r806u11n01 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2518(8127608) in scavenge on r806u11n03 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4803(8157749) in scavenge on r806u14n02 Nov 05 12:37:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7570862_977(7571943) in scavenge on r919u36n04 Nov 05 12:37:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2993(8176783) in scavenge on r815u23n07 Nov 05 12:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2290(8165153) WEXITSTATUS 0 Nov 05 12:37:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2290(8165153) done Nov 05 12:37:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_814(8171981) WEXITSTATUS 0 Nov 05 12:37:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_814(8171981) done Nov 05 12:37:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_812(8171979) WEXITSTATUS 0 Nov 05 12:37:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_812(8171979) done Nov 05 12:37:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_104(8112517) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:37:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3153(8131408) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:37:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_104(8112517) Nov 05 12:37:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3153(8131408) Nov 05 12:37:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_21(8176784) NodeList=r908u37n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:37:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_844(8172018) WEXITSTATUS 0 Nov 05 12:37:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_844(8172018) done Nov 05 12:37:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4048(8176781) WEXITSTATUS 0 Nov 05 12:37:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4048(8176781) done Nov 05 12:37:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:37:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176782 NodeList=r906u41n03 #CPUs=1 Partition=day Nov 05 12:37:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176785 InitPrio=29089 usec=2710 Nov 05 12:37:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176782 WEXITSTATUS 0 Nov 05 12:37:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176782 done Nov 05 12:37:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_616(8171589) WEXITSTATUS 0 Nov 05 12:38:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_616(8171589) done Nov 05 12:38:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_849(8172023) WEXITSTATUS 0 Nov 05 12:38:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_849(8172023) done Nov 05 12:38:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_811(8171978) WEXITSTATUS 0 Nov 05 12:38:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_811(8171978) done Nov 05 12:38:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_843(8172017) WEXITSTATUS 0 Nov 05 12:38:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_843(8172017) done Nov 05 12:38:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_845(8172019) WEXITSTATUS 0 Nov 05 12:38:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_845(8172019) done Nov 05 12:38:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_612(8171585) WEXITSTATUS 0 Nov 05 12:38:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_612(8171585) done Nov 05 12:38:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7447(8166276) WEXITSTATUS 1 Nov 05 12:38:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7447(8166276) done Nov 05 12:38:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_700(8171755) WEXITSTATUS 0 Nov 05 12:38:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_700(8171755) done Nov 05 12:38:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_701(8171756) WEXITSTATUS 0 Nov 05 12:38:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_701(8171756) done Nov 05 12:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_842(8172016) WEXITSTATUS 0 Nov 05 12:38:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_842(8172016) done Nov 05 12:38:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_2353(8126332) has been requeued to reclaim resources for JobId=8176752_22(8176752) Nov 05 12:38:38 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3167(8131626) has been requeued to reclaim resources for JobId=8176752_22(8176752) Nov 05 12:38:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_2353(8126332) Nov 05 12:38:41 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3167(8131626) Nov 05 12:38:41 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_22(8176786) NodeList=r908u37n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:38:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_846(8172020) WEXITSTATUS 0 Nov 05 12:38:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_846(8172020) done Nov 05 12:38:42 r912u23n01 slurmctld[163760]: slurmctld: slurm_job_preempt: preempted JobId=8169688 had to be killed Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:38:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:38:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8160(8172965) WEXITSTATUS 1 Nov 05 12:38:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8160(8172965) done Nov 05 12:38:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:38:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176785 NodeList=r815u23n05 #CPUs=16 Partition=day Nov 05 12:39:12 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1430(8123666) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:39:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3426(8135257) Nov 05 12:39:12 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3606(8136679) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:39:13 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1430(8123666) Nov 05 12:39:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3606(8136679) Nov 05 12:39:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7854(8169623) WEXITSTATUS 1 Nov 05 12:39:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7854(8169623) done Nov 05 12:39:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_23(8176787) NodeList=r908u37n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:39:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_611(8171584) WEXITSTATUS 0 Nov 05 12:39:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_611(8171584) done Nov 05 12:39:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_605(8171578) WEXITSTATUS 0 Nov 05 12:39:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_605(8171578) done Nov 05 12:39:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_608(8171581) WEXITSTATUS 0 Nov 05 12:39:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_608(8171581) done Nov 05 12:39:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_840(8172014) WEXITSTATUS 0 Nov 05 12:39:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_840(8172014) done Nov 05 12:39:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2264(8165084) WEXITSTATUS 0 Nov 05 12:39:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2264(8165084) done Nov 05 12:39:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2258(8165078) WEXITSTATUS 0 Nov 05 12:39:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2258(8165078) done Nov 05 12:39:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_137(8170721) WEXITSTATUS 0 Nov 05 12:39:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_137(8170721) done Nov 05 12:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_609(8171582) WEXITSTATUS 0 Nov 05 12:39:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_609(8171582) done Nov 05 12:39:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_805(8171972) WEXITSTATUS 0 Nov 05 12:39:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_805(8171972) done Nov 05 12:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_809(8171976) WEXITSTATUS 0 Nov 05 12:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_809(8171976) done Nov 05 12:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_808(8171975) WEXITSTATUS 0 Nov 05 12:39:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_808(8171975) done Nov 05 12:39:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_847(8172021) WEXITSTATUS 0 Nov 05 12:39:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_847(8172021) done Nov 05 12:39:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:39:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_509(8114776) WEXITSTATUS 0 Nov 05 12:39:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_509(8114776) done Nov 05 12:39:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_787(8171854) WEXITSTATUS 0 Nov 05 12:39:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_787(8171854) done Nov 05 12:39:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2449(8165723) WEXITSTATUS 0 Nov 05 12:40:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2449(8165723) done Nov 05 12:40:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_24(8176788) in day on r816u03n04 Nov 05 12:40:03 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4049(8176789) in gpu on r903u40n01 Nov 05 12:40:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_121(7572288) WEXITSTATUS 0 Nov 05 12:40:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_121(7572288) done Nov 05 12:40:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_125(7572292) WEXITSTATUS 0 Nov 05 12:40:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_125(7572292) done Nov 05 12:40:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_810(8171977) WEXITSTATUS 0 Nov 05 12:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_810(8171977) done Nov 05 12:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_848(8172022) WEXITSTATUS 0 Nov 05 12:40:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_848(8172022) done Nov 05 12:40:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_613(8171586) WEXITSTATUS 0 Nov 05 12:40:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_613(8171586) done Nov 05 12:40:11 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=7570862_585(7571550) Nov 05 12:40:11 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3421(8135251) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:40:11 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3710(8139219) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:40:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4691(8156463) WEXITSTATUS 1 Nov 05 12:40:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4691(8156463) done Nov 05 12:40:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3710(8139219) Nov 05 12:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_813(8171980) WEXITSTATUS 0 Nov 05 12:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_813(8171980) done Nov 05 12:40:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3421(8135251) Nov 05 12:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_887(8172165) WEXITSTATUS 0 Nov 05 12:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_887(8172165) done Nov 05 12:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_886(8172164) WEXITSTATUS 0 Nov 05 12:40:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_886(8172164) done Nov 05 12:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_118(8170702) WEXITSTATUS 0 Nov 05 12:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_118(8170702) done Nov 05 12:40:17 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8176790 NodeList=(null) usec=1563777 Nov 05 12:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4043(8176742) WEXITSTATUS 0 Nov 05 12:40:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4043(8176742) done Nov 05 12:40:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176790 NodeList=r816u23n09 #CPUs=1 Partition=pi_anticevic Nov 05 12:40:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_25(8176791) NodeList=r908u33n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:40:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4804(8157750) in scavenge on r802u06n03 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3619(8136984) in scavenge on r815u09n11 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4671(8156333) in scavenge on r806u04n02 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4675(8156337) in scavenge on r918u01n01 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2899(8128583) in scavenge on r918u01n01 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3866(8144931) in scavenge on r918u01n03 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4288(8154054) in scavenge on r918u32n04 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4677(8156339) in scavenge on r918u36n04 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5036(8158332) in scavenge on r919u01n03 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5037(8158333) in scavenge on r919u01n03 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5024(8158315) in scavenge on r919u01n04 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5023(8158314) in scavenge on r919u05n02 Nov 05 12:40:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2819(8128404) in scavenge on r919u05n04 Nov 05 12:40:20 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 12:40:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2292(8165155) WEXITSTATUS 0 Nov 05 12:40:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2292(8165155) done Nov 05 12:40:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_607(8171580) WEXITSTATUS 0 Nov 05 12:40:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_607(8171580) done Nov 05 12:40:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_44(8175616) WEXITSTATUS 0 Nov 05 12:40:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_44(8175616) done Nov 05 12:40:38 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_26(8176792) NodeList=r915u29n01 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:40:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3877(8144942) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:40:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3877(8144942) Nov 05 12:40:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7315(8166003) WEXITSTATUS 1 Nov 05 12:40:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7315(8166003) done Nov 05 12:40:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173015 NodeList=r815u03n05 #CPUs=16 Partition=day Nov 05 12:40:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173016 NodeList=r816u03n03 #CPUs=16 Partition=day Nov 05 12:40:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_225(8114491) WEXITSTATUS 0 Nov 05 12:40:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_225(8114491) done Nov 05 12:40:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8057(8172495) WEXITSTATUS 1 Nov 05 12:40:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8057(8172495) done Nov 05 12:41:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8137(8172844) WEXITSTATUS 1 Nov 05 12:41:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8137(8172844) done Nov 05 12:41:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_20(8171551) WEXITSTATUS 0 Nov 05 12:41:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_20(8171551) done Nov 05 12:41:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2373(8165580) WEXITSTATUS 0 Nov 05 12:41:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2373(8165580) done Nov 05 12:41:12 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3792(8143896) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:41:12 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3857(8144801) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_24(8170564) WEXITSTATUS 0 Nov 05 12:41:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_24(8170564) done Nov 05 12:41:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3792(8143896) Nov 05 12:41:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3857(8144801) Nov 05 12:41:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_27(8176793) NodeList=r908u35n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:41:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4028(8175621) WEXITSTATUS 0 Nov 05 12:41:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4028(8175621) done Nov 05 12:41:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_47(8175624) WEXITSTATUS 0 Nov 05 12:41:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_47(8175624) done Nov 05 12:41:25 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_28(8176794) NodeList=r915u29n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:41:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_806(8171973) WEXITSTATUS 0 Nov 05 12:41:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_806(8171973) done Nov 05 12:41:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_610(8171583) WEXITSTATUS 0 Nov 05 12:41:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_610(8171583) done Nov 05 12:41:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_699(8171754) WEXITSTATUS 0 Nov 05 12:41:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_699(8171754) done Nov 05 12:41:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_921(8172285) WEXITSTATUS 0 Nov 05 12:41:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_921(8172285) done Nov 05 12:41:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175194 WEXITSTATUS 1 Nov 05 12:41:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175194 done Nov 05 12:41:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154694 WEXITSTATUS 0 Nov 05 12:41:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154694 done Nov 05 12:41:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:41 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5802(8162761) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:41:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5802(8162761) Nov 05 12:41:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2270(8165101) WEXITSTATUS 0 Nov 05 12:41:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2270(8165101) done Nov 05 12:41:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_972(8172336) WEXITSTATUS 0 Nov 05 12:41:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_972(8172336) done Nov 05 12:41:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173017 NodeList=r913u01n01 #CPUs=16 Partition=day Nov 05 12:41:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173018 NodeList=r816u03n01 #CPUs=16 Partition=day Nov 05 12:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_835(8172009) WEXITSTATUS 0 Nov 05 12:41:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_835(8172009) done Nov 05 12:42:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7532(8166575) WEXITSTATUS 1 Nov 05 12:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7532(8166575) done Nov 05 12:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_807(8171974) WEXITSTATUS 0 Nov 05 12:42:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_807(8171974) done Nov 05 12:42:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_926(8172290) WEXITSTATUS 0 Nov 05 12:42:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_926(8172290) done Nov 05 12:42:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_604(8171577) WEXITSTATUS 0 Nov 05 12:42:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_604(8171577) done Nov 05 12:42:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:12 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3521(8135652) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:42:12 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3931(8145655) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:42:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3521(8135652) Nov 05 12:42:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3931(8145655) Nov 05 12:42:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_500(8171405) WEXITSTATUS 0 Nov 05 12:42:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_500(8171405) done Nov 05 12:42:15 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_29(8176795) NodeList=r908u35n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_962(8172326) WEXITSTATUS 0 Nov 05 12:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_962(8172326) done Nov 05 12:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144162 WEXITSTATUS 0 Nov 05 12:42:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144162 done Nov 05 12:42:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_30(8176796) NodeList=r914u13n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_31(8176797) NodeList=r914u13n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_32(8176798) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_33(8176799) NodeList=r914u13n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_43(8175615) WEXITSTATUS 0 Nov 05 12:42:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_43(8175615) done Nov 05 12:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144163 WEXITSTATUS 0 Nov 05 12:42:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144163 done Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_34(8176800) NodeList=r916u01n03 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_35(8176801) NodeList=r914u13n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_36(8176802) NodeList=r914u13n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_37(8176803) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_38(8176804) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_39(8176805) NodeList=r916u01n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144167 WEXITSTATUS 0 Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144167 done Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_606(8171579) WEXITSTATUS 0 Nov 05 12:42:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_606(8171579) done Nov 05 12:42:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2283(8165132) WEXITSTATUS 0 Nov 05 12:42:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2283(8165132) done Nov 05 12:42:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_40(8176806) NodeList=r908u22n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144168 WEXITSTATUS 0 Nov 05 12:42:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144168 done Nov 05 12:42:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_41(8176807) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_42(8176808) NodeList=r914u09n04 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:42:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_971(8172335) WEXITSTATUS 0 Nov 05 12:42:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_971(8172335) done Nov 05 12:42:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_958(8172322) WEXITSTATUS 0 Nov 05 12:42:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_958(8172322) done Nov 05 12:42:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_974(8172338) WEXITSTATUS 0 Nov 05 12:42:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_974(8172338) done Nov 05 12:42:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_21(8171552) WEXITSTATUS 0 Nov 05 12:42:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_21(8171552) done Nov 05 12:42:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_51(8135750) WEXITSTATUS 0 Nov 05 12:42:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_51(8135750) done Nov 05 12:42:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5125(8158465) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:42:43 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:42:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5125(8158465) Nov 05 12:42:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176739 WEXITSTATUS 0 Nov 05 12:42:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176739 done Nov 05 12:42:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_654(8176809) in pi_ohern on r815u03n11 Nov 05 12:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_655(8176810) in pi_ohern on r815u03n11 Nov 05 12:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173019 in day on r806u14n03 Nov 05 12:42:53 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173022 in day on r816u09n07 Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_43(8176811) in day on r815u03n08 Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4050(8176812) in gpu on r4513u26n01 Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4051(8176813) in gpu on r907u32n01 Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_956(8172320) WEXITSTATUS 0 Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_956(8172320) done Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_890(8172171) WEXITSTATUS 0 Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_890(8172171) done Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_892(8172173) WEXITSTATUS 0 Nov 05 12:42:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_892(8172173) done Nov 05 12:42:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:42:55 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u07n02 Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_128(7572295) WEXITSTATUS 0 Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_128(7572295) done Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_161(7572329) WEXITSTATUS 0 Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_161(7572329) done Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_859(8172118) WEXITSTATUS 0 Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_859(8172118) done Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_861(8172120) WEXITSTATUS 0 Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_861(8172120) done Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_856(8172115) WEXITSTATUS 0 Nov 05 12:42:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_856(8172115) done Nov 05 12:42:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_501(8171406) WEXITSTATUS 0 Nov 05 12:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_501(8171406) done Nov 05 12:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_925(8172289) WEXITSTATUS 0 Nov 05 12:43:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_925(8172289) done Nov 05 12:43:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_216(8170860) WEXITSTATUS 0 Nov 05 12:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_216(8170860) done Nov 05 12:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_951(8172315) WEXITSTATUS 0 Nov 05 12:43:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_951(8172315) done Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4869(8157973) in scavenge on r903u01n01 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4870(8157974) in scavenge on r912u01n01 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4879(8157990) in scavenge on r913u01n01 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4880(8157991) in scavenge on r914u05n04 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4881(8157992) in scavenge on r805u06n02 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4882(8157993) in scavenge on r805u23n02 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4911(8158074) in scavenge on r909u10n04 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5077(8158389) in scavenge on r909u13n04 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4867(8157971) in scavenge on r918u09n04 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4983(8158256) in scavenge on r919u05n03 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5014(8158305) in scavenge on r919u09n02 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_964(8172328) WEXITSTATUS 0 Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_964(8172328) done Nov 05 12:43:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_920(8172284) WEXITSTATUS 0 Nov 05 12:43:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_920(8172284) done Nov 05 12:43:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_959(8172323) WEXITSTATUS 0 Nov 05 12:43:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_959(8172323) done Nov 05 12:43:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2994(8176814) in scavenge on r815u23n07 Nov 05 12:43:12 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176815 InitPrio=24888 usec=6889 Nov 05 12:43:14 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176815 NodeList=r903u01n02 #CPUs=1 Partition=pi_zhu Nov 05 12:43:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4050(8176812) WEXITSTATUS 0 Nov 05 12:43:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4050(8176812) done Nov 05 12:43:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_838(8172012) WEXITSTATUS 0 Nov 05 12:43:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_838(8172012) done Nov 05 12:43:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_966(8172330) WEXITSTATUS 0 Nov 05 12:43:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_966(8172330) done Nov 05 12:43:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8027(8172138) WEXITSTATUS 1 Nov 05 12:43:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8027(8172138) done Nov 05 12:43:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_963(8172327) WEXITSTATUS 0 Nov 05 12:43:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_963(8172327) done Nov 05 12:43:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2389(8165601) WEXITSTATUS 0 Nov 05 12:43:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2389(8165601) done Nov 05 12:43:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_970(8172334) WEXITSTATUS 0 Nov 05 12:43:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_970(8172334) done Nov 05 12:43:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_833(8172007) WEXITSTATUS 0 Nov 05 12:43:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_833(8172007) done Nov 05 12:43:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176815 WEXITSTATUS 1 Nov 05 12:43:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176815 done Nov 05 12:43:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_42(8175612) WEXITSTATUS 0 Nov 05 12:43:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_42(8175612) done Nov 05 12:43:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2363(8165556) WEXITSTATUS 0 Nov 05 12:43:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2363(8165556) done Nov 05 12:43:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5871(8162843) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:43:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5871(8162843) Nov 05 12:43:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_954(8172318) WEXITSTATUS 0 Nov 05 12:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_954(8172318) done Nov 05 12:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_957(8172321) WEXITSTATUS 0 Nov 05 12:43:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_957(8172321) done Nov 05 12:43:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_975(8172339) WEXITSTATUS 0 Nov 05 12:43:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_975(8172339) done Nov 05 12:43:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_952(8172316) WEXITSTATUS 0 Nov 05 12:43:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_952(8172316) done Nov 05 12:43:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_830(8172004) WEXITSTATUS 0 Nov 05 12:43:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_830(8172004) done Nov 05 12:43:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_961(8172325) WEXITSTATUS 0 Nov 05 12:43:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_961(8172325) done Nov 05 12:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_832(8172006) WEXITSTATUS 0 Nov 05 12:43:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_832(8172006) done Nov 05 12:43:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:43:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173020 NodeList=r909u01n01 #CPUs=16 Partition=day Nov 05 12:43:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173021 NodeList=r815u09n08 #CPUs=16 Partition=day Nov 05 12:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_837(8172011) WEXITSTATUS 0 Nov 05 12:43:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_837(8172011) done Nov 05 12:44:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8115(8172790) WEXITSTATUS 1 Nov 05 12:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8115(8172790) done Nov 05 12:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_922(8172286) WEXITSTATUS 0 Nov 05 12:44:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_922(8172286) done Nov 05 12:44:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8118(8172803) WEXITSTATUS 1 Nov 05 12:44:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8118(8172803) done Nov 05 12:44:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_967(8172331) WEXITSTATUS 0 Nov 05 12:44:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_967(8172331) done Nov 05 12:44:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:06 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8176816 NodeList=(null) usec=25231 Nov 05 12:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_917(8172281) WEXITSTATUS 0 Nov 05 12:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_917(8172281) done Nov 05 12:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_919(8172283) WEXITSTATUS 0 Nov 05 12:44:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_919(8172283) done Nov 05 12:44:08 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:44:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:09 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:44:10 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1100 Nov 05 12:44:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8095(8172736) WEXITSTATUS 1 Nov 05 12:44:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8095(8172736) done Nov 05 12:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_923(8172287) WEXITSTATUS 0 Nov 05 12:44:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_923(8172287) done Nov 05 12:44:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169565 WEXITSTATUS 0 Nov 05 12:44:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169565 done Nov 05 12:44:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:44:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_891(8172172) WEXITSTATUS 0 Nov 05 12:44:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_891(8172172) done Nov 05 12:44:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_873(8172146) WEXITSTATUS 0 Nov 05 12:44:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_873(8172146) done Nov 05 12:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_858(8172117) WEXITSTATUS 0 Nov 05 12:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_858(8172117) done Nov 05 12:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_857(8172116) WEXITSTATUS 0 Nov 05 12:44:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_857(8172116) done Nov 05 12:44:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_969(8172333) WEXITSTATUS 0 Nov 05 12:44:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_969(8172333) done Nov 05 12:44:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8141(8172904) WEXITSTATUS 1 Nov 05 12:44:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8141(8172904) done Nov 05 12:44:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_960(8172324) WEXITSTATUS 0 Nov 05 12:44:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_960(8172324) done Nov 05 12:44:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_924(8172288) WEXITSTATUS 0 Nov 05 12:44:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_924(8172288) done Nov 05 12:44:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8145(8172908) WEXITSTATUS 1 Nov 05 12:44:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8145(8172908) done Nov 05 12:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_950(8172314) WEXITSTATUS 0 Nov 05 12:44:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_950(8172314) done Nov 05 12:44:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_853(8172112) WEXITSTATUS 0 Nov 05 12:44:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_853(8172112) done Nov 05 12:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_953(8172317) WEXITSTATUS 0 Nov 05 12:44:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_953(8172317) done Nov 05 12:44:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_955(8172319) WEXITSTATUS 0 Nov 05 12:44:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_955(8172319) done Nov 05 12:44:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_902(8172183) WEXITSTATUS 0 Nov 05 12:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_902(8172183) done Nov 05 12:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_898(8172179) WEXITSTATUS 0 Nov 05 12:44:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_898(8172179) done Nov 05 12:44:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:40 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176817 InitPrio=21146 usec=5045 Nov 05 12:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_836(8172010) WEXITSTATUS 0 Nov 05 12:44:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_836(8172010) done Nov 05 12:44:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_854(8172113) WEXITSTATUS 0 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_854(8172113) done Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176785 WEXITSTATUS 0 Nov 05 12:44:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176785 done Nov 05 12:44:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4051(8176813) WEXITSTATUS 0 Nov 05 12:44:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4051(8176813) done Nov 05 12:44:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:44:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176816 NodeList=r909u01n01 #CPUs=1 Partition=day Nov 05 12:44:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173023 NodeList=r815u23n05 #CPUs=16 Partition=day Nov 05 12:44:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176817 NodeList=r804u06n[03-04],r804u09n[01-04],r804u11n[01-02] #CPUs=192 Partition=mpi Nov 05 12:44:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_918(8172282) WEXITSTATUS 0 Nov 05 12:44:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_918(8172282) done Nov 05 12:44:57 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176818 InitPrio=59397 usec=3525 Nov 05 12:44:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_968(8172332) WEXITSTATUS 0 Nov 05 12:44:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_968(8172332) done Nov 05 12:44:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_965(8172329) WEXITSTATUS 0 Nov 05 12:45:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_965(8172329) done Nov 05 12:45:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_900(8172181) WEXITSTATUS 0 Nov 05 12:45:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_900(8172181) done Nov 05 12:45:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_877(8172155) WEXITSTATUS 0 Nov 05 12:45:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_877(8172155) done Nov 05 12:45:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_449(8171245) WEXITSTATUS 0 Nov 05 12:45:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_449(8171245) done Nov 05 12:45:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_901(8172182) WEXITSTATUS 0 Nov 05 12:45:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_901(8172182) done Nov 05 12:45:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_119(8114385) Nov 05 12:45:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_137(8114403) Nov 05 12:45:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_170(8114436) Nov 05 12:45:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_227(8114493) Nov 05 12:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_831(8172005) WEXITSTATUS 0 Nov 05 12:45:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_831(8172005) done Nov 05 12:45:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4881(8157992) WEXITSTATUS 1 Nov 05 12:45:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4881(8157992) done Nov 05 12:45:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_894(8172175) WEXITSTATUS 0 Nov 05 12:45:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_894(8172175) done Nov 05 12:45:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_860(8172119) WEXITSTATUS 0 Nov 05 12:45:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_860(8172119) done Nov 05 12:45:19 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176819 InitPrio=21138 usec=2595 Nov 05 12:45:21 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176819 NodeList=r806u25n01 #CPUs=1 Partition=devel Nov 05 12:45:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_834(8172008) WEXITSTATUS 0 Nov 05 12:45:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_834(8172008) done Nov 05 12:45:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8117(8172792) WEXITSTATUS 1 Nov 05 12:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8117(8172792) done Nov 05 12:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_973(8172337) WEXITSTATUS 0 Nov 05 12:45:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_973(8172337) done Nov 05 12:45:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7760(8168914) WEXITSTATUS 1 Nov 05 12:45:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7760(8168914) done Nov 05 12:45:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176818 in day on r909u10n04 Nov 05 12:45:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_44(8176820) in day on r815u03n03 Nov 05 12:45:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_45(8176821) in day on r815u15n08 Nov 05 12:45:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4052(8176822) in gpu on r4513u26n01 Nov 05 12:45:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4053(8176823) in gpu on r907u32n01 Nov 05 12:45:44 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2995(8176824) in day on r815u23n05 Nov 05 12:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_937(8172301) WEXITSTATUS 0 Nov 05 12:45:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_937(8172301) done Nov 05 12:45:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_186(7572354) WEXITSTATUS 0 Nov 05 12:45:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_186(7572354) done Nov 05 12:45:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:47 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176825 InitPrio=25500 usec=3504 Nov 05 12:45:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144212 in scavenge on r807u39n01,r807u41n01 Nov 05 12:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_899(8172180) WEXITSTATUS 0 Nov 05 12:45:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_899(8172180) done Nov 05 12:45:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6668(8164754) WEXITSTATUS 1 Nov 05 12:45:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6668(8164754) done Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4884(8157995) in scavenge on r802u04n04 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5100(8158433) in scavenge on r903u01n02 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4861(8157960) in scavenge on r911u09n01 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4728(8156857) in scavenge on r904u01n01 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4209(8152047) in scavenge on r806u11n02 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4709(8156592) in scavenge on r913u13n03 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2058(8125845) in scavenge on r914u01n02 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4812(8157769) in scavenge on r914u01n02 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5004(8158288) in scavenge on r918u05n04 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4269(8153751) in scavenge on r919u01n01 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3445(8135293) in scavenge on r805u06n02 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_191(7572359) in scavenge on r805u09n01 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_895(8172176) WEXITSTATUS 0 Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_895(8172176) done Nov 05 12:45:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4076(8148135) WEXITSTATUS 1 Nov 05 12:46:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4076(8148135) done Nov 05 12:46:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_874(8172152) WEXITSTATUS 0 Nov 05 12:46:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_874(8172152) done Nov 05 12:46:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_852(8172111) WEXITSTATUS 0 Nov 05 12:46:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_852(8172111) done Nov 05 12:46:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_23(8171554) WEXITSTATUS 0 Nov 05 12:46:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_23(8171554) done Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_904(8172185) WEXITSTATUS 0 Nov 05 12:46:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_904(8172185) done Nov 05 12:46:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_905(8172186) WEXITSTATUS 0 Nov 05 12:46:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_905(8172186) done Nov 05 12:46:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_928(8172292) WEXITSTATUS 0 Nov 05 12:46:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_928(8172292) done Nov 05 12:46:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_850(8172109) WEXITSTATUS 0 Nov 05 12:46:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_850(8172109) done Nov 05 12:46:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4053(8176823) WEXITSTATUS 0 Nov 05 12:46:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4053(8176823) done Nov 05 12:46:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_851(8172110) WEXITSTATUS 0 Nov 05 12:46:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_851(8172110) done Nov 05 12:46:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_933(8172297) WEXITSTATUS 0 Nov 05 12:46:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_933(8172297) done Nov 05 12:46:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2319(8165273) WEXITSTATUS 0 Nov 05 12:46:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2319(8165273) done Nov 05 12:46:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7451(8166280) WEXITSTATUS 1 Nov 05 12:46:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7451(8166280) done Nov 05 12:46:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2434(8165683) WEXITSTATUS 0 Nov 05 12:46:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2434(8165683) done Nov 05 12:46:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_942(8172306) WEXITSTATUS 0 Nov 05 12:46:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_942(8172306) done Nov 05 12:46:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_936(8172300) WEXITSTATUS 0 Nov 05 12:46:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_936(8172300) done Nov 05 12:46:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_941(8172305) WEXITSTATUS 0 Nov 05 12:46:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_941(8172305) done Nov 05 12:46:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_893(8172174) WEXITSTATUS 0 Nov 05 12:46:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_893(8172174) done Nov 05 12:46:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_929(8172293) WEXITSTATUS 0 Nov 05 12:46:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_929(8172293) done Nov 05 12:46:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2330(8165346) WEXITSTATUS 0 Nov 05 12:46:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2330(8165346) done Nov 05 12:46:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_264(7572432) WEXITSTATUS 0 Nov 05 12:46:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_264(7572432) done Nov 05 12:46:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:46:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_896(8172177) WEXITSTATUS 0 Nov 05 12:46:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_896(8172177) done Nov 05 12:47:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_931(8172295) WEXITSTATUS 0 Nov 05 12:47:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_931(8172295) done Nov 05 12:47:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4052(8176822) WEXITSTATUS 0 Nov 05 12:47:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4052(8176822) done Nov 05 12:47:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8168728 Nov 05 12:47:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_839(8172013) WEXITSTATUS 0 Nov 05 12:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_839(8172013) done Nov 05 12:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_943(8172307) WEXITSTATUS 0 Nov 05 12:47:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_943(8172307) done Nov 05 12:47:14 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176826 InitPrio=72402 usec=6707 Nov 05 12:47:16 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176826 NodeList=r806u25n01 #CPUs=1 Partition=devel Nov 05 12:47:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4047(8176780) WEXITSTATUS 0 Nov 05 12:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4047(8176780) done Nov 05 12:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_944(8172308) WEXITSTATUS 0 Nov 05 12:47:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_944(8172308) done Nov 05 12:47:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_948(8172312) WEXITSTATUS 0 Nov 05 12:47:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_948(8172312) done Nov 05 12:47:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_930(8172294) WEXITSTATUS 0 Nov 05 12:47:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_930(8172294) done Nov 05 12:47:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_946(8172310) WEXITSTATUS 0 Nov 05 12:47:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_946(8172310) done Nov 05 12:47:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_947(8172311) WEXITSTATUS 0 Nov 05 12:47:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_947(8172311) done Nov 05 12:47:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_940(8172304) WEXITSTATUS 0 Nov 05 12:47:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_940(8172304) done Nov 05 12:47:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_875(8172153) WEXITSTATUS 0 Nov 05 12:47:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_875(8172153) done Nov 05 12:47:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_939(8172303) WEXITSTATUS 0 Nov 05 12:47:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_939(8172303) done Nov 05 12:47:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_989(8172356) WEXITSTATUS 0 Nov 05 12:47:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_989(8172356) done Nov 05 12:47:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:47:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173024 NodeList=r816u03n07 #CPUs=16 Partition=day Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135457 WEXITSTATUS 0 Nov 05 12:48:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135457 done Nov 05 12:48:03 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u25n04 Nov 05 12:48:03 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u27n01 Nov 05 12:48:03 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u25n03 Nov 05 12:48:03 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u27n02 Nov 05 12:48:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8035(8172416) WEXITSTATUS 1 Nov 05 12:48:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8035(8172416) done Nov 05 12:48:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_945(8172309) WEXITSTATUS 0 Nov 05 12:48:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_945(8172309) done Nov 05 12:48:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8175193 Nov 05 12:48:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_935(8172299) WEXITSTATUS 0 Nov 05 12:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_935(8172299) done Nov 05 12:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_85(8171736) WEXITSTATUS 0 Nov 05 12:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_85(8171736) done Nov 05 12:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_82(8171733) WEXITSTATUS 0 Nov 05 12:48:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_82(8171733) done Nov 05 12:48:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2482(8165837) WEXITSTATUS 0 Nov 05 12:48:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2482(8165837) done Nov 05 12:48:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_906(8172187) WEXITSTATUS 0 Nov 05 12:48:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_906(8172187) done Nov 05 12:48:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_855(8172114) WEXITSTATUS 0 Nov 05 12:48:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_855(8172114) done Nov 05 12:48:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_993(8172360) WEXITSTATUS 0 Nov 05 12:48:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_993(8172360) done Nov 05 12:48:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_656(8176827) in pi_ohern on r815u03n11 Nov 05 12:48:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_657(8176828) in pi_ohern on r815u03n11 Nov 05 12:48:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_658(8176829) in pi_ohern on r815u03n12 Nov 05 12:48:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173025 in day on r909u01n01 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_46(8176830) in day on r904u07n02 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_47(8176831) in day on r918u01n02 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176752_48(8176832) in day on r816u09n08 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4054(8176833) in gpu on r4513u20n01 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4055(8176834) in gpu on r4513u26n01 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4056(8176835) in gpu on r907u32n01 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r904u04n01 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2996(8176836) in day on r815u09n06 Nov 05 12:48:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2997(8176837) in day on r815u15n02 Nov 05 12:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_481(8114748) WEXITSTATUS 0 Nov 05 12:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_481(8114748) done Nov 05 12:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_487(7572655) WEXITSTATUS 0 Nov 05 12:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_487(7572655) done Nov 05 12:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_368(7572536) WEXITSTATUS 0 Nov 05 12:48:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_368(7572536) done Nov 05 12:48:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4880(8157991) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_876(8172154) WEXITSTATUS 0 Nov 05 12:48:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_876(8172154) done Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4762(8157333) in scavenge on r802u09n03 Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2054(8125841) in scavenge on r807u03n02 Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2403(8126438) in scavenge on r904u10n01 Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4446(8155759) in scavenge on r912u32n02 Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_197(7572365) in scavenge on r913u13n01 Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_938(7573107) in scavenge on r913u24n03 Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4880(8157991) Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_89(8171740) WEXITSTATUS 0 Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_89(8171740) done Nov 05 12:48:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176838 InitPrio=34916 usec=6835 Nov 05 12:48:50 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176838 NodeList=r806u25n02 #CPUs=3 Partition=devel Nov 05 12:48:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:48:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_938(8172302) WEXITSTATUS 0 Nov 05 12:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_938(8172302) done Nov 05 12:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_48(8175627) WEXITSTATUS 0 Nov 05 12:49:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_48(8175627) done Nov 05 12:49:06 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176752_49(8176752) NodeList=r915u29n02 #CPUs=16 Partition=pi_hammes_schiffer Nov 05 12:49:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7822(8169459) WEXITSTATUS 1 Nov 05 12:49:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7822(8169459) done Nov 05 12:49:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7981(8171129) WEXITSTATUS 1 Nov 05 12:49:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7981(8171129) done Nov 05 12:49:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_186(8171956) WEXITSTATUS 0 Nov 05 12:49:12 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_186(8171956) done Nov 05 12:49:13 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7816(8169418) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:49:13 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7912(8170211) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:49:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_995(8172362) WEXITSTATUS 0 Nov 05 12:49:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_995(8172362) done Nov 05 12:49:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8136(8172843) WEXITSTATUS 1 Nov 05 12:49:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8136(8172843) done Nov 05 12:49:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7816(8169418) Nov 05 12:49:15 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7912(8170211) Nov 05 12:49:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176839 InitPrio=51140 usec=7676 Nov 05 12:49:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176839 NodeList=r903u37n01 #CPUs=1 Partition=gpu_devel Nov 05 12:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8119(8172804) WEXITSTATUS 1 Nov 05 12:49:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8119(8172804) done Nov 05 12:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_88(8171739) WEXITSTATUS 0 Nov 05 12:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_88(8171739) done Nov 05 12:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_932(8172296) WEXITSTATUS 0 Nov 05 12:49:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_932(8172296) done Nov 05 12:49:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_996(8172363) WEXITSTATUS 0 Nov 05 12:49:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_996(8172363) done Nov 05 12:49:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7890(8170008) WEXITSTATUS 1 Nov 05 12:49:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7890(8170008) done Nov 05 12:49:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_994(8172361) WEXITSTATUS 0 Nov 05 12:49:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_994(8172361) done Nov 05 12:49:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_999(8170528) WEXITSTATUS 0 Nov 05 12:49:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_999(8170528) done Nov 05 12:49:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2300(8165186) WEXITSTATUS 0 Nov 05 12:49:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2300(8165186) done Nov 05 12:49:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_991(8172358) WEXITSTATUS 0 Nov 05 12:49:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_991(8172358) done Nov 05 12:49:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5757(8162622) WEXITSTATUS 1 Nov 05 12:49:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5757(8162622) done Nov 05 12:49:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_988(8172355) WEXITSTATUS 0 Nov 05 12:49:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_988(8172355) done Nov 05 12:49:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:49:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175219 NodeList=r815u03n07 #CPUs=24 Partition=day Nov 05 12:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_934(8172298) WEXITSTATUS 0 Nov 05 12:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_934(8172298) done Nov 05 12:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_46(8165439) WEXITSTATUS 0 Nov 05 12:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_46(8165439) done Nov 05 12:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8144(8172907) WEXITSTATUS 1 Nov 05 12:49:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8144(8172907) done Nov 05 12:49:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176840 InitPrio=24888 usec=4664 Nov 05 12:50:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176840 NodeList=r903u01n02 #CPUs=1 Partition=pi_zhu Nov 05 12:50:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:50:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_927(8172291) WEXITSTATUS 0 Nov 05 12:50:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_927(8172291) done Nov 05 12:50:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_992(8172359) WEXITSTATUS 0 Nov 05 12:50:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_992(8172359) done Nov 05 12:50:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_22(8171553) WEXITSTATUS 0 Nov 05 12:50:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_22(8171553) done Nov 05 12:50:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_195(8171965) WEXITSTATUS 0 Nov 05 12:50:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_195(8171965) done Nov 05 12:50:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176840 WEXITSTATUS 1 Nov 05 12:50:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176840 done Nov 05 12:50:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:28 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8176841 NodeList=(null) usec=20190 Nov 05 12:50:29 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176841 NodeList=r806u23n03 #CPUs=1 Partition=devel Nov 05 12:50:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_987(8172354) WEXITSTATUS 0 Nov 05 12:50:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_987(8172354) done Nov 05 12:50:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8128(8172835) WEXITSTATUS 1 Nov 05 12:50:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8128(8172835) done Nov 05 12:50:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_188(8171958) WEXITSTATUS 0 Nov 05 12:50:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_188(8171958) done Nov 05 12:50:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:42 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_477(8114744) Nov 05 12:50:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6067(8163090) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:50:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_6710(8164815) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:50:42 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7208(8165848) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:50:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_179(8171949) WEXITSTATUS 0 Nov 05 12:50:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_179(8171949) done Nov 05 12:50:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7208(8165848) Nov 05 12:50:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6710(8164815) Nov 05 12:50:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_6067(8163090) Nov 05 12:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_990(8172357) WEXITSTATUS 0 Nov 05 12:50:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_990(8172357) done Nov 05 12:50:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4056(8176835) WEXITSTATUS 0 Nov 05 12:50:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4056(8176835) done Nov 05 12:50:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:50:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175617 NodeList=r918u05n02 #CPUs=24 Partition=day Nov 05 12:51:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_998(8172366) WEXITSTATUS 0 Nov 05 12:51:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_998(8172366) done Nov 05 12:51:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2442(8165716) WEXITSTATUS 0 Nov 05 12:51:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2442(8165716) done Nov 05 12:51:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_659(8176842) in pi_ohern on r815u03n11 Nov 05 12:51:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_660(8176843) in pi_ohern on r815u03n11 Nov 05 12:51:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_661(8176844) in pi_ohern on r815u03n11 Nov 05 12:51:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_662(8176845) in pi_ohern on r815u03n11 Nov 05 12:51:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_663(8176846) in pi_ohern on r815u03n12 Nov 05 12:51:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_664(8176847) in pi_ohern on r815u03n12 Nov 05 12:51:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_665(8176848) in pi_ohern on r815u03n12 Nov 05 12:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4057(8176849) in gpu on r907u32n01 Nov 05 12:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2998(8176850) in day on r914u05n04 Nov 05 12:51:22 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2999(8176851) in day on r815u23n01 Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_184(8171954) WEXITSTATUS 0 Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_184(8171954) done Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_664(8176847) WEXITSTATUS 0 Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_664(8176847) done Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_663(8176846) WEXITSTATUS 0 Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_663(8176846) done Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_661(8176844) WEXITSTATUS 0 Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_661(8176844) done Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_660(8176843) WEXITSTATUS 0 Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_660(8176843) done Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_662(8176845) WEXITSTATUS 0 Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_662(8176845) done Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_665(8176848) WEXITSTATUS 0 Nov 05 12:51:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_665(8176848) done Nov 05 12:51:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1989(8125568) in scavenge on r916u04n01 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3408(8135235) in scavenge on r815u03n10 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4842(8157850) in scavenge on r806u11n04 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2297(8126245) in scavenge on r914u05n04 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5018(8158309) in scavenge on r913u09n03 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_566(8113946) in scavenge on r913u13n04 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2625(8127883) in scavenge on r909u01n01 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3444(8135292) in scavenge on r909u07n02 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2413(8126655) in scavenge on r909u07n04 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3461(8135329) in scavenge on r805u02n01 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3594(8136426) in scavenge on r805u06n02 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5468(8159891) in scavenge on r805u09n02 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5470(8159893) in scavenge on r805u11n01 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_672(8114941) in scavenge on r815u15n03 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_199(7572367) in scavenge on r816u23n05 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3738(8141698) WEXITSTATUS 1 Nov 05 12:51:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3738(8141698) done Nov 05 12:51:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2469(8165780) in scavenge on r816u23n08 Nov 05 12:51:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2461(8165762) in scavenge on r816u23n05 Nov 05 12:51:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2465(8165776) in scavenge on r816u23n05 Nov 05 12:51:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2684(8166485) in scavenge on r816u23n05 Nov 05 12:51:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 12:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_997(8172365) WEXITSTATUS 0 Nov 05 12:51:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_997(8172365) done Nov 05 12:51:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4151(8150167) WEXITSTATUS 1 Nov 05 12:51:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4151(8150167) done Nov 05 12:51:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:43 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7747(8168850) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:51:43 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_8093(8172680) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:51:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8190(8112304) WEXITSTATUS 1 Nov 05 12:51:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8190(8112304) done Nov 05 12:51:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7747(8168850) Nov 05 12:51:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_8093(8172680) Nov 05 12:51:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154978 WEXITSTATUS 0 Nov 05 12:51:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154978 done Nov 05 12:51:51 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u27n02.grace.ycrc.yale.edu" Nov 05 12:51:51 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u27n02.grace.ycrc.yale.edu" Nov 05 12:51:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_182(8171952) WEXITSTATUS 0 Nov 05 12:51:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_182(8171952) done Nov 05 12:51:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_889(8172170) WEXITSTATUS 0 Nov 05 12:51:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_889(8172170) done Nov 05 12:51:53 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u27n01.grace.ycrc.yale.edu" Nov 05 12:51:53 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u27n01.grace.ycrc.yale.edu" Nov 05 12:51:53 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u25n04.grace.ycrc.yale.edu" Nov 05 12:51:53 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u25n04.grace.ycrc.yale.edu" Nov 05 12:51:54 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u25n03.grace.ycrc.yale.edu" Nov 05 12:51:54 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u25n03.grace.ycrc.yale.edu" Nov 05 12:51:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_986(8172353) WEXITSTATUS 0 Nov 05 12:51:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_986(8172353) done Nov 05 12:51:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8175618 NodeList=r806u04n01 #CPUs=24 Partition=day Nov 05 12:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_178(8171948) WEXITSTATUS 0 Nov 05 12:51:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_178(8171948) done Nov 05 12:51:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:51:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7929(8170453) WEXITSTATUS 1 Nov 05 12:51:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7929(8170453) done Nov 05 12:52:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:52:02 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:52:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8165(8174005) WEXITSTATUS 1 Nov 05 12:52:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8165(8174005) done Nov 05 12:52:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_39(8175481) WEXITSTATUS 0 Nov 05 12:52:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8174123_39(8175481) done Nov 05 12:52:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:13 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3448(8135302) Nov 05 12:52:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_190(8171960) WEXITSTATUS 0 Nov 05 12:52:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_190(8171960) done Nov 05 12:52:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_888(8172169) WEXITSTATUS 0 Nov 05 12:52:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_888(8172169) done Nov 05 12:52:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_174(8171944) WEXITSTATUS 0 Nov 05 12:52:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_174(8171944) done Nov 05 12:52:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_255(8172077) WEXITSTATUS 0 Nov 05 12:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_255(8172077) done Nov 05 12:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7962(8170796) WEXITSTATUS 1 Nov 05 12:52:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7962(8170796) done Nov 05 12:52:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:52:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176852 InitPrio=24888 usec=4655 Nov 05 12:52:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176852 NodeList=r903u01n02 #CPUs=1 Partition=pi_zhu Nov 05 12:52:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_819(8171993) WEXITSTATUS 0 Nov 05 12:53:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_819(8171993) done Nov 05 12:53:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8114217_398(8114665) Nov 05 12:53:12 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3450(8135304) Nov 05 12:53:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7186(8165811) WEXITSTATUS 1 Nov 05 12:53:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7186(8165811) done Nov 05 12:53:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4269(8153751) WEXITSTATUS 1 Nov 05 12:53:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4269(8153751) done Nov 05 12:53:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2251(8165055) WEXITSTATUS 0 Nov 05 12:53:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2251(8165055) done Nov 05 12:53:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:35 r912u23n01 slurmctld[163760]: slurmctld: node r804u27n02 returned to service Nov 05 12:53:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169566 WEXITSTATUS 0 Nov 05 12:53:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169566 done Nov 05 12:53:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2487(8165877) WEXITSTATUS 0 Nov 05 12:53:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2487(8165877) done Nov 05 12:53:38 r912u23n01 slurmctld[163760]: slurmctld: node r804u25n04 returned to service Nov 05 12:53:38 r912u23n01 slurmctld[163760]: slurmctld: node r804u27n01 returned to service Nov 05 12:53:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:39 r912u23n01 slurmctld[163760]: slurmctld: node r804u25n03 returned to service Nov 05 12:53:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176852 WEXITSTATUS 0 Nov 05 12:53:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176852 done Nov 05 12:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2371(8165578) WEXITSTATUS 0 Nov 05 12:53:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2371(8165578) done Nov 05 12:53:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7469(8166337) WEXITSTATUS 1 Nov 05 12:53:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7469(8166337) done Nov 05 12:53:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2281(8165125) WEXITSTATUS 0 Nov 05 12:53:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2281(8165125) done Nov 05 12:53:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_773(8115042) WEXITSTATUS 0 Nov 05 12:53:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_773(8115042) done Nov 05 12:53:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176853 InitPrio=48613 usec=3033 Nov 05 12:54:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_367(7572535) WEXITSTATUS 0 Nov 05 12:54:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_367(7572535) done Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:54:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_666(8176854) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_667(8176855) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169127_668(8169127) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_1(8176856) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_2(8176857) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_3(8176858) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_4(8176859) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_5(8176860) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_6(8176861) in pi_ohern on r815u03n11 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_7(8176862) in pi_ohern on r815u03n12 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_8(8176863) in pi_ohern on r815u03n12 Nov 05 12:54:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_9(8176864) in pi_ohern on r815u03n12 Nov 05 12:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_187(8171957) WEXITSTATUS 0 Nov 05 12:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_187(8171957) done Nov 05 12:54:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176853 NodeList=r806u28n01,r806u31n01 #CPUs=2 Partition=bigmem Nov 05 12:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2276(8165120) WEXITSTATUS 0 Nov 05 12:54:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2276(8165120) done Nov 05 12:54:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 12:54:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3000(8176865) in day on r904u04n01 Nov 05 12:54:06 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3001(8176866) in day on r919u01n01 Nov 05 12:54:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:900 Nov 05 12:54:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 12:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_668(8169127) WEXITSTATUS 0 Nov 05 12:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_668(8169127) done Nov 05 12:54:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_666(8176854) WEXITSTATUS 0 Nov 05 12:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_666(8176854) done Nov 05 12:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_667(8176855) WEXITSTATUS 0 Nov 05 12:54:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_667(8176855) done Nov 05 12:54:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_173(8171943) WEXITSTATUS 0 Nov 05 12:54:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_173(8171943) done Nov 05 12:54:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:13 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:100 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2144(8125964) in scavenge on r911u24n02 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2407(8126446) in scavenge on r916u04n02 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3149(8131207) in scavenge on r802u02n04 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3073(8130388) in scavenge on r802u09n02 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4122(8149428) in scavenge on r815u09n12 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4768(8157450) in scavenge on r909u13n03 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4520(8155921) in scavenge on r919u01n01 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3990(8146254) in scavenge on r919u01n01 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4970(8158232) in scavenge on r806u09n02 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5312(8158840) in scavenge on r806u11n04 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5349(8159083) in scavenge on r806u11n04 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4973(8158235) in scavenge on r914u05n01 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5227(8158630) in scavenge on r914u05n04 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_784(7572953) in scavenge on r815u23n10 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_754(8171820) WEXITSTATUS 0 Nov 05 12:54:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_754(8171820) done Nov 05 12:54:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:18 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 12:54:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2939(8171686) in scavenge on r816u23n08 Nov 05 12:54:18 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2374(8165581) in scavenge on r816u23n11 Nov 05 12:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_32(8171563) WEXITSTATUS 0 Nov 05 12:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_32(8171563) done Nov 05 12:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7827(8169520) WEXITSTATUS 1 Nov 05 12:54:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7827(8169520) done Nov 05 12:54:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_258(8172080) WEXITSTATUS 0 Nov 05 12:54:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_258(8172080) done Nov 05 12:54:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8166(8174006) WEXITSTATUS 1 Nov 05 12:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8166(8174006) done Nov 05 12:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7805(8169348) WEXITSTATUS 1 Nov 05 12:54:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7805(8169348) done Nov 05 12:54:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2273(8165117) WEXITSTATUS 0 Nov 05 12:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2273(8165117) done Nov 05 12:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_185(8171955) WEXITSTATUS 0 Nov 05 12:54:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_185(8171955) done Nov 05 12:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7448(8166277) WEXITSTATUS 1 Nov 05 12:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7448(8166277) done Nov 05 12:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2488(8165878) WEXITSTATUS 0 Nov 05 12:54:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2488(8165878) done Nov 05 12:54:39 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:43 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4289(8154055) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:54:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4289(8154055) Nov 05 12:54:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154724 WEXITSTATUS 0 Nov 05 12:54:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154724 done Nov 05 12:54:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:54:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_87(8171738) WEXITSTATUS 0 Nov 05 12:54:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_87(8171738) done Nov 05 12:54:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7630(8167080) WEXITSTATUS 1 Nov 05 12:55:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7630(8167080) done Nov 05 12:55:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_86(8171737) WEXITSTATUS 0 Nov 05 12:55:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_86(8171737) done Nov 05 12:55:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_2(8176755) WEXITSTATUS 0 Nov 05 12:55:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_2(8176755) done Nov 05 12:55:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_253(8172075) WEXITSTATUS 0 Nov 05 12:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_253(8172075) done Nov 05 12:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175617 WEXITSTATUS 0 Nov 05 12:55:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175617 done Nov 05 12:55:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_180(8171950) WEXITSTATUS 0 Nov 05 12:55:11 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_180(8171950) done Nov 05 12:55:13 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:13 r912u23n01 slurmctld[163760]: slurmctld: sched: _slurm_rpc_allocate_resources JobId=8176867 NodeList=(null) usec=67943 Nov 05 12:55:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_9(8176762) WEXITSTATUS 0 Nov 05 12:55:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_9(8176762) done Nov 05 12:55:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2507(8165969) WEXITSTATUS 0 Nov 05 12:55:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2507(8165969) done Nov 05 12:55:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3853(8144682) WEXITSTATUS 1 Nov 05 12:55:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3853(8144682) done Nov 05 12:55:14 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 12:55:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176867 NodeList=r806u25n01 #CPUs=1 Partition=devel Nov 05 12:55:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_4(8176757) WEXITSTATUS 0 Nov 05 12:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_4(8176757) done Nov 05 12:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_3(8176756) WEXITSTATUS 0 Nov 05 12:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_3(8176756) done Nov 05 12:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_254(8172076) WEXITSTATUS 0 Nov 05 12:55:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_254(8172076) done Nov 05 12:55:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_10(8176763) WEXITSTATUS 0 Nov 05 12:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_10(8176763) done Nov 05 12:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4306(8154072) WEXITSTATUS 1 Nov 05 12:55:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4306(8154072) done Nov 05 12:55:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_6(8176759) WEXITSTATUS 0 Nov 05 12:55:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_6(8176759) done Nov 05 12:55:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8071(8172579) WEXITSTATUS 1 Nov 05 12:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8071(8172579) done Nov 05 12:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_7(8176760) WEXITSTATUS 0 Nov 05 12:55:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_7(8176760) done Nov 05 12:55:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_177(8171947) WEXITSTATUS 0 Nov 05 12:55:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_177(8171947) done Nov 05 12:55:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_193(8171963) WEXITSTATUS 0 Nov 05 12:55:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_193(8171963) done Nov 05 12:55:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2327(8165343) WEXITSTATUS 0 Nov 05 12:55:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2327(8165343) done Nov 05 12:55:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_175(8171945) WEXITSTATUS 0 Nov 05 12:55:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_175(8171945) done Nov 05 12:55:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:43 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5103(8158436) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:55:43 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_7926(8170450) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:55:43 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_8114(8172789) has been requeued to reclaim resources for JobId=Invalid Nov 05 12:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_346(8172247) WEXITSTATUS 0 Nov 05 12:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_346(8172247) done Nov 05 12:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_344(8172245) WEXITSTATUS 0 Nov 05 12:55:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_344(8172245) done Nov 05 12:55:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_199(8171969) WEXITSTATUS 0 Nov 05 12:55:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_199(8171969) done Nov 05 12:55:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5103(8158436) Nov 05 12:55:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_8114(8172789) Nov 05 12:55:45 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_7926(8170450) Nov 05 12:55:45 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176868 InitPrio=24841 usec=4476 Nov 05 12:55:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_259(8172081) WEXITSTATUS 0 Nov 05 12:55:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_259(8172081) done Nov 05 12:55:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_5(8176758) WEXITSTATUS 0 Nov 05 12:55:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_5(8176758) done Nov 05 12:55:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_257(8172079) WEXITSTATUS 0 Nov 05 12:55:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_257(8172079) done Nov 05 12:55:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:55:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176825 NodeList=r805u04n02 #CPUs=24 Partition=day Nov 05 12:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_262(8172084) WEXITSTATUS 0 Nov 05 12:55:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_262(8172084) done Nov 05 12:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176825 WEXITSTATUS 0 Nov 05 12:55:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176825 done Nov 05 12:55:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_194(8171964) WEXITSTATUS 0 Nov 05 12:56:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_194(8171964) done Nov 05 12:56:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:56:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:56:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8076(8172584) WEXITSTATUS 1 Nov 05 12:56:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8076(8172584) done Nov 05 12:56:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_196(8171966) WEXITSTATUS 0 Nov 05 12:56:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_196(8171966) done Nov 05 12:56:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_13(8176766) WEXITSTATUS 0 Nov 05 12:56:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_13(8176766) done Nov 05 12:56:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_181(8171951) WEXITSTATUS 0 Nov 05 12:56:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_181(8171951) done Nov 05 12:56:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144150 WEXITSTATUS 0 Nov 05 12:56:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144150 done Nov 05 12:56:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175618 WEXITSTATUS 0 Nov 05 12:56:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175618 done Nov 05 12:56:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4049(8176789) WEXITSTATUS 0 Nov 05 12:56:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4049(8176789) done Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4337(8154947) has been requeued to reclaim resources for JobId=8176868 Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_5611(8161712) has been requeued to reclaim resources for JobId=8176868 Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3549(8135844) has been requeued to reclaim resources for JobId=8176868 Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3786(8143789) has been requeued to reclaim resources for JobId=8176868 Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8147100 has been requeued to reclaim resources for JobId=8176868 Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8147101 has been requeued to reclaim resources for JobId=8176868 Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7222(8165870) WEXITSTATUS 1 Nov 05 12:56:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7222(8165870) done Nov 05 12:56:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8003(8171677) WEXITSTATUS 1 Nov 05 12:56:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8003(8171677) done Nov 05 12:56:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3786(8143789) Nov 05 12:56:42 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_5611(8161712) Nov 05 12:56:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144121 WEXITSTATUS 0 Nov 05 12:56:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8144121 done Nov 05 12:56:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3549(8135844) Nov 05 12:56:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8147101 Nov 05 12:56:43 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4337(8154947) Nov 05 12:56:44 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8147100 Nov 05 12:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_0(8176753) WEXITSTATUS 0 Nov 05 12:56:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_0(8176753) done Nov 05 12:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_26(8171557) WEXITSTATUS 0 Nov 05 12:56:45 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_26(8171557) done Nov 05 12:56:46 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8176868 NodeList=r903u04n04 #CPUs=48 Partition=pi_zhu Nov 05 12:56:46 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_10(8176869) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_11(8176870) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_12(8176871) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_13(8176872) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_14(8176873) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_15(8176874) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_16(8176875) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_17(8176876) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_18(8176877) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_19(8176878) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_20(8176879) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_21(8176880) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_22(8176881) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_23(8176882) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_24(8176883) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_25(8176884) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_26(8176885) in pi_ohern on r815u03n11 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_27(8176886) in pi_ohern on r815u03n12 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_28(8176887) in pi_ohern on r815u03n12 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_29(8176888) in pi_ohern on r815u03n12 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_30(8176889) in pi_ohern on r815u03n12 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_31(8176890) in pi_ohern on r815u03n12 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_32(8176891) in pi_ohern on r815u03n12 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_33(8176892) in pi_ohern on r815u03n12 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_34(8176893) in pi_ohern on r807u22n01 Nov 05 12:56:48 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_35(8176894) in pi_ohern on r911u32n01 Nov 05 12:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4058(8176895) in gpu on r903u40n01 Nov 05 12:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3002(8176896) in day on r805u04n02 Nov 05 12:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3003(8176897) in day on r806u04n01 Nov 05 12:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3004(8176898) in day on r807u08n02 Nov 05 12:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3005(8176899) in day on r904u10n03 Nov 05 12:56:50 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3006(8176900) in day on r918u05n02 Nov 05 12:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8068(8172576) WEXITSTATUS 1 Nov 05 12:56:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8068(8172576) done Nov 05 12:56:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_176(8171946) WEXITSTATUS 0 Nov 05 12:56:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_176(8171946) done Nov 05 12:56:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_11(8176764) WEXITSTATUS 0 Nov 05 12:56:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_11(8176764) done Nov 05 12:56:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_28(8171559) WEXITSTATUS 0 Nov 05 12:56:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_28(8171559) done Nov 05 12:56:57 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4992(8158271) in scavenge on r801u14n03 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4200(8152038) in scavenge on r802u02n01 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2924(8128608) in scavenge on r802u09n04 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2932(8128616) in scavenge on r802u11n01 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5339(8158958) in scavenge on r815u03n10 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5408(8159389) in scavenge on r815u29n12 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3945(8145777) in scavenge on r816u29n09 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4357(8154967) in scavenge on r909u01n04 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4688(8156350) in scavenge on r914u05n04 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3834(8144448) in scavenge on r807u05n04 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5622(8161723) in scavenge on r807u11n01 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5558(8161048) in scavenge on r806u04n01 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5620(8161721) in scavenge on r806u04n01 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4098(8148578) in scavenge on r806u04n01 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4532(8155933) in scavenge on r806u11n01 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3884(8144949) in scavenge on r806u11n03 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4617(8156196) in scavenge on r806u11n03 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5185(8158539) in scavenge on r913u01n02 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5556(8161046) in scavenge on r913u05n01 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5575(8161229) in scavenge on r913u28n02 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_925(7573094) in scavenge on r805u04n02 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_911(8115180) in scavenge on r918u05n02 Nov 05 12:56:59 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_796(7572965) in scavenge on r815u23n05 Nov 05 12:57:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_191(8171961) WEXITSTATUS 0 Nov 05 12:57:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_191(8171961) done Nov 05 12:57:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2596(8166233) in scavenge on r816u23n11 Nov 05 12:57:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2419(8165659) in scavenge on r806u31n01 Nov 05 12:57:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2710(8166545) in scavenge on r806u31n01 Nov 05 12:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4055(8176834) WEXITSTATUS 0 Nov 05 12:57:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4055(8176834) done Nov 05 12:57:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2302(8165188) WEXITSTATUS 0 Nov 05 12:57:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2302(8165188) done Nov 05 12:57:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8152(8172957) WEXITSTATUS 1 Nov 05 12:57:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8152(8172957) done Nov 05 12:57:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2261(8165081) WEXITSTATUS 0 Nov 05 12:57:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2261(8165081) done Nov 05 12:57:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7956(8170790) WEXITSTATUS 1 Nov 05 12:57:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7956(8170790) done Nov 05 12:57:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2426(8165666) WEXITSTATUS 0 Nov 05 12:57:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2426(8165666) done Nov 05 12:57:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_348(8172249) WEXITSTATUS 0 Nov 05 12:57:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_348(8172249) done Nov 05 12:57:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_29(8171560) WEXITSTATUS 0 Nov 05 12:57:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_29(8171560) done Nov 05 12:57:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_264(8172086) WEXITSTATUS 0 Nov 05 12:57:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_264(8172086) done Nov 05 12:57:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172968 WEXITSTATUS 0 Nov 05 12:57:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172968 done Nov 05 12:57:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176901 InitPrio=10612 usec=2131 Nov 05 12:57:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8176902 InitPrio=10612 usec=2768 Nov 05 12:57:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3646(8137583) WEXITSTATUS 1 Nov 05 12:57:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3646(8137583) done Nov 05 12:57:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175650 WEXITSTATUS 0 Nov 05 12:57:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175650 done Nov 05 12:57:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_0(8165305) WEXITSTATUS 0 Nov 05 12:57:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_0(8165305) done Nov 05 12:57:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2339(8165430) WEXITSTATUS 0 Nov 05 12:57:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2339(8165430) done Nov 05 12:57:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8120(8172805) WEXITSTATUS 1 Nov 05 12:57:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8120(8172805) done Nov 05 12:57:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:48 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7829889 uid 22319 Nov 05 12:57:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:57:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8173026 NodeList=r815u15n07 #CPUs=16 Partition=day Nov 05 12:57:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_20(8176779) WEXITSTATUS 0 Nov 05 12:57:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_20(8176779) done Nov 05 12:57:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 12:58:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 12:58:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=7824297 uid 22319 Nov 05 12:58:05 r912u23n01 slurmctld[163760]: slurmctld: job_str_signal(3): invalid JobId=7824297 Nov 05 12:58:05 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: job_str_signal() uid=22319 JobId=7824297 sig=9 returned: Invalid job id specified Nov 05 12:58:15 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8112304_3459(8135321) Nov 05 12:58:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_341(8172242) WEXITSTATUS 0 Nov 05 12:58:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_341(8172242) done Nov 05 12:58:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2696(8128067) WEXITSTATUS 1 Nov 05 12:58:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_2696(8128067) done Nov 05 12:58:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_192(8171962) WEXITSTATUS 0 Nov 05 12:58:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_192(8171962) done Nov 05 12:58:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175460 WEXITSTATUS 0 Nov 05 12:58:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175460 done Nov 05 12:58:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_183(8171953) WEXITSTATUS 0 Nov 05 12:58:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_183(8171953) done Nov 05 12:58:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3961(8146004) WEXITSTATUS 1 Nov 05 12:58:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3961(8146004) done Nov 05 12:58:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8038(8172419) WEXITSTATUS 1 Nov 05 12:58:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8038(8172419) done Nov 05 12:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2494(8165907) WEXITSTATUS 0 Nov 05 12:58:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2494(8165907) done Nov 05 12:58:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_11(8165339) WEXITSTATUS 0 Nov 05 12:58:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_11(8165339) done Nov 05 12:58:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:58:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176868 WEXITSTATUS 1 Nov 05 12:58:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176868 done Nov 05 12:59:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_260(8172082) WEXITSTATUS 0 Nov 05 12:59:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_260(8172082) done Nov 05 12:59:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_189(8171959) WEXITSTATUS 0 Nov 05 12:59:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_189(8171959) done Nov 05 12:59:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_345(8172246) WEXITSTATUS 0 Nov 05 12:59:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_345(8172246) done Nov 05 12:59:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7524(8166543) WEXITSTATUS 1 Nov 05 12:59:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7524(8166543) done Nov 05 12:59:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_30(8171561) WEXITSTATUS 0 Nov 05 12:59:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_30(8171561) done Nov 05 12:59:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2475(8165806) WEXITSTATUS 0 Nov 05 12:59:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2475(8165806) done Nov 05 12:59:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_36(8176903) in pi_ohern on r815u03n11 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_37(8176904) in pi_ohern on r815u03n11 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_38(8176905) in pi_ohern on r815u03n11 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_39(8176906) in pi_ohern on r815u03n11 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_40(8176907) in pi_ohern on r815u03n12 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_41(8176908) in pi_ohern on r815u03n12 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_42(8176909) in pi_ohern on r815u03n12 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_43(8176910) in pi_ohern on r815u03n12 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_44(8176911) in pi_ohern on r815u03n12 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_45(8176912) in pi_ohern on r815u03n12 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_46(8176913) in pi_ohern on r807u22n01 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_47(8176914) in pi_ohern on r903u12n01 Nov 05 12:59:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_48(8176915) in pi_ohern on r903u12n01 Nov 05 12:59:34 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4059(8176916) in gpu on r4513u26n01 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r906u41n03 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3007(8176917) in day on r909u13n01 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3008(8176918) in day on r913u05n04 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3009(8176919) in day on r918u01n03 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3010(8176920) in day on r815u29n04 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3011(8176921) in day on r815u09n04 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3012(8176922) in day on r815u09n06 Nov 05 12:59:35 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3013(8176923) in day on r815u15n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_1(8176924) in day on r906u41n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_2(8176925) in day on r906u41n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_3(8176926) in day on r906u41n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_4(8176927) in day on r906u41n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_5(8176928) in day on r906u41n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_6(8176929) in day on r904u04n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_7(8176930) in day on r904u04n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_8(8176931) in day on r904u04n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_9(8176932) in day on r904u04n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_10(8176933) in day on r904u04n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_11(8176934) in day on r904u04n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_12(8176935) in day on r904u04n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_13(8176936) in day on r904u04n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_14(8176937) in day on r904u04n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_15(8176938) in day on r904u04n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_16(8176939) in day on r904u10n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_17(8176940) in day on r904u10n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_18(8176941) in day on r904u10n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_19(8176942) in day on r904u10n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_20(8176943) in day on r904u10n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_21(8176944) in day on r904u10n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_22(8176945) in day on r904u10n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_23(8176946) in day on r904u10n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_24(8176947) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_25(8176948) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_26(8176949) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_27(8176950) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_28(8176951) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_29(8176952) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_30(8176953) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_31(8176954) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_32(8176955) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_33(8176956) in day on r904u10n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_34(8176957) in day on r914u01n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_35(8176958) in day on r914u01n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_36(8176959) in day on r914u01n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_37(8176960) in day on r914u01n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_38(8176961) in day on r914u01n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_39(8176962) in day on r914u01n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_40(8176963) in day on r914u05n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_41(8176964) in day on r914u05n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_42(8176965) in day on r914u05n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_43(8176966) in day on r914u05n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_44(8176967) in day on r914u05n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_45(8176968) in day on r914u05n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_46(8176969) in day on r914u05n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_47(8176970) in day on r807u03n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_48(8176971) in day on r807u03n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_49(8176972) in day on r807u03n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_50(8176973) in day on r807u03n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_51(8176974) in day on r807u03n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_52(8176975) in day on r807u03n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_53(8176976) in day on r807u03n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_54(8176977) in day on r807u03n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_55(8176978) in day on r807u03n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_56(8176979) in day on r807u03n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_57(8176980) in day on r807u03n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_58(8176981) in day on r807u05n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_59(8176982) in day on r807u05n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_60(8176983) in day on r807u05n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_61(8176984) in day on r807u05n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_62(8176985) in day on r807u05n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_63(8176986) in day on r807u05n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_64(8176987) in day on r807u05n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_65(8176988) in day on r807u05n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_66(8176989) in day on r807u05n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_67(8176990) in day on r807u05n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_68(8176991) in day on r807u05n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_69(8176992) in day on r807u05n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_70(8176993) in day on r807u05n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_71(8176994) in day on r807u05n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_72(8176995) in day on r807u08n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_73(8176996) in day on r807u08n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_74(8176997) in day on r807u08n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_75(8176998) in day on r807u08n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_76(8176999) in day on r807u08n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_77(8177000) in day on r807u08n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_78(8177001) in day on r807u08n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_79(8177002) in day on r807u08n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_80(8177003) in day on r807u08n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_81(8177004) in day on r807u08n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_82(8177005) in day on r807u08n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_83(8177006) in day on r807u08n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_84(8177007) in day on r807u11n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_85(8177008) in day on r807u11n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_86(8177009) in day on r807u11n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_87(8177010) in day on r807u11n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_88(8177011) in day on r807u11n01 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_89(8177012) in day on r807u11n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_90(8177013) in day on r807u11n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_91(8177014) in day on r807u11n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_92(8177015) in day on r807u11n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_93(8177016) in day on r807u11n02 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_94(8177017) in day on r807u11n03 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_95(8177018) in day on r807u11n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_96(8177019) in day on r807u11n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_97(8177020) in day on r807u11n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_98(8177021) in day on r807u11n04 Nov 05 12:59:36 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_99(8177022) in day on r807u11n04 Nov 05 12:59:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_100(8177023) in day on r912u28n01 Nov 05 12:59:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8147101 in scavenge on r908u39n03 Nov 05 12:59:37 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8144182 in scavenge on r903u04n04,r916u01n02 Nov 05 12:59:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_27(8171558) WEXITSTATUS 0 Nov 05 12:59:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_27(8171558) done Nov 05 12:59:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4752(8157102) in scavenge on r801u30n03 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4924(8158111) in scavenge on r908u39n03 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5545(8160880) in scavenge on r802u02n01 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3929(8145653) in scavenge on r911u24n03 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_1521(8123784) in scavenge on r915u38n03 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3064(8129875) in scavenge on r816u03n09 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2106(8125917) in scavenge on r815u03n10 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3959(8146002) in scavenge on r815u15n09 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4734(8156863) in scavenge on r816u23n09 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_937(7573106) in scavenge on r816u23n01 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_267(8114533) in scavenge on r816u23n01 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_466(7572634) in scavenge on r816u23n02 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_933(7573102) in scavenge on r816u23n04 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_595(8114862) in scavenge on r816u29n01 Nov 05 12:59:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_798(8115067) in scavenge on r816u29n02 Nov 05 12:59:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4058(8176895) WEXITSTATUS 0 Nov 05 12:59:44 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4058(8176895) done Nov 05 12:59:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8171687 Nov 05 12:59:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_1(8177024) in scavenge on r801u30n01 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_2(8177025) in scavenge on r908u28n02 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_3(8177026) in scavenge on r908u28n02 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_4(8177027) in scavenge on r908u33n03 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_5(8177028) in scavenge on r908u37n01 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_6(8177029) in scavenge on r908u37n02 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_7(8177030) in scavenge on r908u37n03 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_8(8177031) in scavenge on r908u37n04 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_9(8177032) in scavenge on r908u39n02 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176901_10(8176901) in scavenge on r907u01n02 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_2754(8166781) in scavenge on r816u23n02 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_860(8127974) in scavenge on r816u23n02 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _attempt_backfill: 150 pending RPCs at cycle end, consider configuring max_rpc_cnt Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_101(8171752) WEXITSTATUS 0 Nov 05 12:59:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_101(8171752) done Nov 05 12:59:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7908(8170207) WEXITSTATUS 1 Nov 05 12:59:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7908(8170207) done Nov 05 12:59:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177033 InitPrio=10612 usec=1921 Nov 05 12:59:55 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177034 InitPrio=10612 usec=2948 Nov 05 12:59:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 12:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6008(8163013) WEXITSTATUS 1 Nov 05 12:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6008(8163013) done Nov 05 12:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175649 WEXITSTATUS 0 Nov 05 12:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175649 done Nov 05 12:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2274(8165118) WEXITSTATUS 0 Nov 05 12:59:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2274(8165118) done Nov 05 12:59:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 13:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 13:00:01 r912u23n01 slurmctld[163760]: slurmctld: update_node: expansion of node specification '' resulted in zero nodes Nov 05 13:00:01 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_update_node for : Invalid node name specified Nov 05 13:00:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Updated reservation=maintenance accounts=admins nodes=r801u02n[01-04],r801u04n[01-04],r801u06n[01-04],r801u09n[01-04],r801u11n[01-04],r801u14n[01-04],r801u23n[01-04],r801u25n[01-04],r801u27n[01-04],r801u30n[01-04],r801u32n[01-04],r802u02n[01-04],r802u04n[01-04],r802u06n[01-04],r802u09n[01-04],r802u11n[01-04],r802u14n[01-04],r802u23n[01-04],r802u25n[01-04],r802u27n[01-04],r802u30n[01-04],r802u32n[01-04],r803u02n[01-04],r803u04n[01-04],r803u06n[01-04],r803u09n[01-04],r803u11n[01-04],r803u14n[01-04],r803u23n[01-04],r803u25n[01-04],r803u27n[01-04],r803u30n[01-04],r803u32n[01-04],r804u02n[01-04],r804u04n[01-04],r804u06n[01-04],r804u09n[01-04],r804u11n[01-04],r804u14n[01-04],r804u23n[01-04],r804u25n[01-04],r804u27n[01-04],r804u30n[01-04],r804u32n[01-04],r805u02n[01-04],r805u04n[01-04],r805u06n[01-04],r805u09n[01-04],r805u11n[01-04],r805u14n[01-04],r805u23n[01-04],r805u25n[01-04],r805u27n[01-04],r805u30n01,r805u32n01,r805u34n01,r805u36n01,r805u38n01,r805u40n01,r806u02n[01-04],r806u04n[01-04],r806u06n[01-04],r806u09n[01-04],r806u11n[01-04],r806u14n[01-04],r806u23n[01-04],r806u25n[01-03],r806u28n01,r806u31n01,r806u33n01,r806u35n01,r806u37n01,r806u39n01,r806u41n01,r807u01n[01-04],r807u03n[01-04],r807u05n[01-04],r807u08n[01-04],r807u11n[01-04],r807u14n01,r807u22n01,r807u24n01,r807u26n01,r807u31n01,r807u35n01,r807u37n01,r807u39n01,r807u41n01,r808u08n01,r808u11n01,r808u14n01,r808u17n01,r814u29n[05,07],r815u03n[01-12],r815u09n[01-12],r815u15n[01-12],r815u23n[01-12],r815u29n[01-12],r815u35n[01-07,09-12],r816u03n[01-12],r816u09n[01-12],r816u15n[01-12],r816u23n[01-12],r816u29n[01-12],r816u35n[01-07,09-12],r902u01n01,r902u04n01,r902u07n01,r902u10n01,r903u01n[01-04],r903u04n[01-04],r903u07n[01-04],r903u12n01,r903u14n01,r903u24n01,r903u26n01,r903u30n01,r903u31n01,r903u34n01,r903u37n01,r903u40n01,r904u01n[01-04],r904u04n[01-04],r904u07n[01-04],r904u10n[01-04],r904u13n01,r905u01n[01-04],r905u05n[01-04],r905u09n[01-04],r905u13n[01-04],r905u24n[01-04],r905u28n[01-04],r905u32n[01-04],r905u36n[01-04],r906u01n[01-04],r906u05n[01-04],r906u09n[01-04],r906u13n01,r906u25n01,r906u28n01,r906u32n01,r906u35n01,r906u37n01,r906u39n01,r906u41n[01-04],r907u01n[01-04],r907u05n[01-04],r907u09n[01-04],r907u24n01,r907u28n01,r907u32n01,r907u34n01,r907u36n01,r907u38n01,r907u40n01,r908u22n[01-02],r908u24n[01-02],r908u28n[01-03],r908u33n[01-04],r908u35n[01-04],r908u37n[01-04],r908u39n[01-04],r909u01n[01-04],r909u04n[01-04],r909u07n[01-04],r909u10n[01-04],r909u13n[01-04],r909u29n[01-04],r911u01n[01-04],r911u05n[01-04],r911u09n[01-04],r911u13n[01-04],r911u24n[01-04],r911u28n[01-02],r911u32n01,r912u01n[01-04],r912u05n[01-04],r912u09n01,r912u13n01,r912u28n[01-04],r912u32n[01-04],r912u36n[01-04],r913u01n[01-04],r913u05n[01-04],r913u09n[01-04],r913u13n[01-04],r913u24n[01-04],r913u28n[01-04],r913u32n01,r913u35n01,r914u01n[01-04],r914u05n[01-04],r914u09n[01-04],r914u13n[01-04],r914u30n01,r914u36n01,r915u01n01,r915u05n01,r915u09n01,r915u13n01,r915u29n[01-04],r915u32n[01-04],r915u35n[01-04],r915u38n[01-04],r916u01n[01-04],r916u04n[01-04],r917u01n[01-04],r917u04n01,r917u07n[01-04],r917u10n[01-04],r917u13n[01-04],r917u23n01,r917u29n[01-04],r918u01n[01-04],r918u05n[01-04],r918u09n[01-04],r918u32n[01-04],r918u36n[01-04],r919u01n[01-04],r919u05n[01-04],r919u09n[01-04],r919u32n[01-04],r919u36n[01-04],r4512u31n01,r4512u32n01,r4513u20n01,r4513u23n01,r4513u26n01,r4513u29n01,r4513u32n01 cores=29212 licenses=(null) tres=cpu=30172 watts=4294967294 start=2024-12-03T08:00:00 end=2024-12-10T08:00:00 MaxStartDelay= Comment= Nov 05 13:00:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_343(8172244) WEXITSTATUS 0 Nov 05 13:00:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_343(8172244) done Nov 05 13:00:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_347(8172248) WEXITSTATUS 0 Nov 05 13:00:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_347(8172248) done Nov 05 13:00:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176816 WEXITSTATUS 0 Nov 05 13:00:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176816 done Nov 05 13:00:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_96(8171747) WEXITSTATUS 0 Nov 05 13:00:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_96(8171747) done Nov 05 13:00:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:17 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177035 InitPrio=24841 usec=2963 Nov 05 13:00:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_172(8171942) WEXITSTATUS 0 Nov 05 13:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_172(8171942) done Nov 05 13:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169474_4(8169479) WEXITSTATUS 143 Nov 05 13:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169474_4(8169479) done Nov 05 13:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_64(7571028) WEXITSTATUS 0 Nov 05 13:00:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7570862_64(7571028) done Nov 05 13:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_100(8171751) WEXITSTATUS 0 Nov 05 13:00:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_100(8171751) done Nov 05 13:00:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_198(8171968) WEXITSTATUS 0 Nov 05 13:00:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_198(8171968) done Nov 05 13:00:22 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_35(8171566) WEXITSTATUS 0 Nov 05 13:00:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_35(8171566) done Nov 05 13:00:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2297(8165172) WEXITSTATUS 0 Nov 05 13:00:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2297(8165172) done Nov 05 13:00:25 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176817 WEXITSTATUS 0 Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176817 done Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n04 Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n01 Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n04 Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u11n02 Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u11n01 Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n03 Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u09n02 Nov 05 13:00:27 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r804u06n03 Nov 05 13:00:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_84(8171735) WEXITSTATUS 0 Nov 05 13:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_84(8171735) done Nov 05 13:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_31(8171562) WEXITSTATUS 0 Nov 05 13:00:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_31(8171562) done Nov 05 13:00:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_263(8172085) WEXITSTATUS 0 Nov 05 13:00:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_263(8172085) done Nov 05 13:00:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_949(8172313) WEXITSTATUS 0 Nov 05 13:00:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_949(8172313) done Nov 05 13:00:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_256(8172078) WEXITSTATUS 0 Nov 05 13:00:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_256(8172078) done Nov 05 13:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_33(8171564) WEXITSTATUS 0 Nov 05 13:00:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_33(8171564) done Nov 05 13:00:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2427(8165667) WEXITSTATUS 0 Nov 05 13:00:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2427(8165667) done Nov 05 13:00:47 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 13:00:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_197(8171967) WEXITSTATUS 0 Nov 05 13:00:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_197(8171967) done Nov 05 13:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_40(8171571) WEXITSTATUS 0 Nov 05 13:00:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_40(8171571) done Nov 05 13:00:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=6321884 NodeList=r807u03n01 #CPUs=1 Partition=day Nov 05 13:00:55 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=7357650 NodeList=r807u03n01 #CPUs=1 Partition=day Nov 05 13:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_37(8171568) WEXITSTATUS 0 Nov 05 13:00:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_37(8171568) done Nov 05 13:00:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_34(8171565) WEXITSTATUS 0 Nov 05 13:00:56 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_34(8171565) done Nov 05 13:00:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2489(8165879) WEXITSTATUS 0 Nov 05 13:01:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2489(8165879) done Nov 05 13:01:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7791(8169268) WEXITSTATUS 1 Nov 05 13:01:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7791(8169268) done Nov 05 13:01:06 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_43(8176811) WEXITSTATUS 0 Nov 05 13:01:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_43(8176811) done Nov 05 13:01:09 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3842(8144567) has been requeued to reclaim resources for JobId=8177035 Nov 05 13:01:09 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_4041(8147094) has been requeued to reclaim resources for JobId=8177035 Nov 05 13:01:09 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_366(8113245) has been requeued to reclaim resources for JobId=8177035 Nov 05 13:01:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_4041(8147094) Nov 05 13:01:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3842(8144567) Nov 05 13:01:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_366(8113245) Nov 05 13:01:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177035 NodeList=r903u01n03 #CPUs=48 Partition=pi_zhu Nov 05 13:01:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_95(8171746) WEXITSTATUS 0 Nov 05 13:01:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_95(8171746) done Nov 05 13:01:18 r912u23n01 slurmctld[163760]: slurmctld: build_job_queue has run for 2000000 usec, exiting with 200 of 12976 jobs tested, 16 job-partition pairs added Nov 05 13:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 WEXITSTATUS 0 Nov 05 13:01:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7357650 done Nov 05 13:01:22 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=7357650 Nov 05 13:01:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_872(8172145) WEXITSTATUS 0 Nov 05 13:01:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_872(8172145) done Nov 05 13:01:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8177035 WEXITSTATUS 2 Nov 05 13:01:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8177035 done Nov 05 13:01:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7828(8169521) WEXITSTATUS 1 Nov 05 13:01:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7828(8169521) done Nov 05 13:01:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2740(8166757) WEXITSTATUS 0 Nov 05 13:01:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2740(8166757) done Nov 05 13:01:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_69(8135768) WEXITSTATUS 0 Nov 05 13:01:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135693_69(8135768) done Nov 05 13:01:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8097(8172738) WEXITSTATUS 1 Nov 05 13:01:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8097(8172738) done Nov 05 13:01:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_3(8165330) WEXITSTATUS 0 Nov 05 13:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_3(8165330) done Nov 05 13:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_41(8171572) WEXITSTATUS 0 Nov 05 13:01:57 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_41(8171572) done Nov 05 13:01:58 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:01:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_94(8171745) WEXITSTATUS 0 Nov 05 13:01:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_94(8171745) done Nov 05 13:02:01 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154722 WEXITSTATUS 0 Nov 05 13:02:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154722 done Nov 05 13:02:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_49(8177036) in pi_ohern on r815u03n11 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_50(8177037) in pi_ohern on r815u03n11 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_51(8177038) in pi_ohern on r815u03n11 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_52(8177039) in pi_ohern on r815u03n12 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_53(8177040) in pi_ohern on r815u03n12 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_54(8177041) in pi_ohern on r815u03n12 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_55(8177042) in pi_ohern on r815u03n12 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_56(8177043) in pi_ohern on r815u03n12 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_57(8177044) in pi_ohern on r903u12n01 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_58(8177045) in pi_ohern on r911u32n01 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_59(8177046) in pi_ohern on r911u32n01 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_60(8177047) in pi_ohern on r911u32n01 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_61(8177048) in pi_ohern on r807u22n01 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_62(8177049) in pi_ohern on r807u22n01 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_63(8177050) in pi_ohern on r807u22n01 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_64(8177051) in pi_ohern on r807u24n01 Nov 05 13:02:16 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_65(8177052) in pi_ohern on r913u32n01 Nov 05 13:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_66(8177053) in pi_ohern on r913u32n01 Nov 05 13:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_67(8177054) in pi_ohern on r913u32n01 Nov 05 13:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_68(8177055) in pi_ohern on r913u35n01 Nov 05 13:02:17 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_69(8177056) in pi_ohern on r913u35n01 Nov 05 13:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4057(8176849) WEXITSTATUS 0 Nov 05 13:02:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4057(8176849) done Nov 05 13:02:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4060(8177057) in gpu on r903u40n01 Nov 05 13:02:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3014(8177058) in day on r806u23n01 Nov 05 13:02:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3015(8177059) in day on r913u01n02 Nov 05 13:02:19 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3016(8177060) in day on r815u29n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_242(8114508) WEXITSTATUS 0 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_242(8114508) done Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_101(8177061) in day on r906u41n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_102(8177062) in day on r904u04n02 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_103(8177063) in day on r904u04n02 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_104(8177064) in day on r912u28n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_105(8177065) in day on r912u28n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_106(8177066) in day on r912u36n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_107(8177067) in day on r912u36n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_108(8177068) in day on r912u36n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_109(8177069) in day on r912u36n02 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_110(8177070) in day on r912u36n02 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_111(8177071) in day on r912u36n02 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_112(8177072) in day on r912u36n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_113(8177073) in day on r912u36n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_114(8177074) in day on r912u36n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_115(8177075) in day on r912u36n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_116(8177076) in day on r912u36n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_117(8177077) in day on r912u36n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_118(8177078) in day on r912u36n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_119(8177079) in day on r912u36n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_120(8177080) in day on r913u01n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_121(8177081) in day on r913u01n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_122(8177082) in day on r913u01n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_123(8177083) in day on r913u01n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_124(8177084) in day on r913u05n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_125(8177085) in day on r913u05n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_126(8177086) in day on r913u05n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_127(8177087) in day on r913u05n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_128(8177088) in day on r913u05n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_129(8177089) in day on r913u05n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_130(8177090) in day on r913u09n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_131(8177091) in day on r913u09n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_132(8177092) in day on r913u09n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_133(8177093) in day on r913u09n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_134(8177094) in day on r913u13n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_135(8177095) in day on r913u13n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_136(8177096) in day on r913u13n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_137(8177097) in day on r913u13n03 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_138(8177098) in day on r913u13n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_139(8177099) in day on r913u13n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_140(8177100) in day on r913u13n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_141(8177101) in day on r913u13n04 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_142(8177102) in day on r913u24n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_143(8177103) in day on r913u24n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_144(8177104) in day on r913u24n01 Nov 05 13:02:20 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_145(8177105) in day on r913u24n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_146(8177106) in day on r913u24n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_147(8177107) in day on r913u24n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_148(8177108) in day on r913u28n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_149(8177109) in day on r909u01n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_150(8177110) in day on r909u01n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_151(8177111) in day on r909u01n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_152(8177112) in day on r909u01n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_153(8177113) in day on r909u01n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_154(8177114) in day on r909u01n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_155(8177115) in day on r909u01n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_156(8177116) in day on r909u01n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_157(8177117) in day on r909u01n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_158(8177118) in day on r909u01n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_159(8177119) in day on r909u04n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_160(8177120) in day on r909u04n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_161(8177121) in day on r909u04n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_162(8177122) in day on r909u07n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_163(8177123) in day on r909u07n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_164(8177124) in day on r909u07n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_165(8177125) in day on r909u07n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_166(8177126) in day on r909u07n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_167(8177127) in day on r909u07n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_168(8177128) in day on r909u07n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_169(8177129) in day on r909u10n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_170(8177130) in day on r909u10n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_171(8177131) in day on r909u10n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_172(8177132) in day on r909u10n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_173(8177133) in day on r909u10n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_174(8177134) in day on r909u10n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_175(8177135) in day on r909u13n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_176(8177136) in day on r909u13n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_177(8177137) in day on r909u13n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_178(8177138) in day on r909u13n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_179(8177139) in day on r909u13n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_180(8177140) in day on r909u13n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_181(8177141) in day on r909u13n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_182(8177142) in day on r909u13n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_183(8177143) in day on r909u13n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_184(8177144) in day on r909u29n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_185(8177145) in day on r909u29n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_186(8177146) in day on r918u01n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_187(8177147) in day on r918u01n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_188(8177148) in day on r918u01n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_189(8177149) in day on r918u01n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_190(8177150) in day on r918u01n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_191(8177151) in day on r918u01n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_192(8177152) in day on r918u01n03 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_193(8177153) in day on r918u05n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_194(8177154) in day on r918u05n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_195(8177155) in day on r918u05n04 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_196(8177156) in day on r918u09n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_197(8177157) in day on r918u09n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_198(8177158) in day on r918u09n01 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_199(8177159) in day on r918u09n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_200(8177160) in day on r918u09n02 Nov 05 13:02:21 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8147100 in scavenge on r908u33n04 Nov 05 13:02:22 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:112 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 13:02:24 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177161 InitPrio=34224 usec=5052 Nov 05 13:02:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4299(8154065) in scavenge on r908u33n04 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2900(8128584) in scavenge on r801u06n03 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4715(8156726) in scavenge on r801u25n01 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4900(8158033) in scavenge on r816u29n11 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_3507(8135534) in scavenge on r806u23n01 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5697(8162359) in scavenge on r918u09n02 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5715(8162480) in scavenge on r919u01n01 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4471(8155788) in scavenge on r805u14n01 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_280(7572448) in scavenge on r815u03n08 Nov 05 13:02:27 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_697(7572866) in scavenge on r815u03n08 Nov 05 13:02:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_1(8177162) in scavenge on r907u01n02 Nov 05 13:02:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_2(8177163) in scavenge on r907u01n02 Nov 05 13:02:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176743 WEXITSTATUS 0 Nov 05 13:02:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176743 done Nov 05 13:02:51 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6374(8164045) WEXITSTATUS 1 Nov 05 13:02:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_6374(8164045) done Nov 05 13:02:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177161 NodeList=r807u03n01 #CPUs=4 Partition=day Nov 05 13:02:56 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177164 InitPrio=27624 usec=3549 Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_0(8177165) NodeList=r907u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_1(8177166) NodeList=r907u01n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_2(8177167) NodeList=r907u01n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_3(8177168) NodeList=r907u01n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_4(8177169) NodeList=r907u01n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_5(8177170) NodeList=r907u01n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_6(8177171) NodeList=r907u01n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_7(8177172) NodeList=r907u01n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_8(8177173) NodeList=r907u01n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_9(8177174) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_10(8177175) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_11(8177176) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_12(8177177) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_13(8177178) NodeList=r907u05n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_14(8177179) NodeList=r907u05n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_15(8177180) NodeList=r907u05n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_16(8177181) NodeList=r907u05n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_17(8177182) NodeList=r907u05n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_18(8177183) NodeList=r907u05n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_19(8177184) NodeList=r907u05n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_20(8177185) NodeList=r907u05n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_21(8177186) NodeList=r907u05n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_22(8177187) NodeList=r907u05n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_23(8177188) NodeList=r907u05n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_24(8177189) NodeList=r907u05n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_25(8177190) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_26(8177191) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_27(8177192) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_28(8177193) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_29(8177194) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_30(8177195) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_31(8177196) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_32(8177197) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_33(8177198) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_34(8177199) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_35(8177200) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_36(8177201) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_37(8177202) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_38(8177203) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_39(8177204) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_40(8177205) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_41(8177206) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_42(8177207) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_43(8177208) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_44(8177209) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_45(8177210) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_46(8177211) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_47(8177212) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_48(8177213) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_184(7572352) WEXITSTATUS 0 Nov 05 13:02:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_184(7572352) done Nov 05 13:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2460(8165750) WEXITSTATUS 0 Nov 05 13:03:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2460(8165750) done Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_49(8177214) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_50(8177215) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_51(8177216) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_52(8177217) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_53(8177218) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_54(8177219) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_55(8177220) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_56(8177221) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_399(7572567) WEXITSTATUS 0 Nov 05 13:03:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_399(7572567) done Nov 05 13:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_201(8171971) WEXITSTATUS 0 Nov 05 13:03:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_201(8171971) done Nov 05 13:03:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2481(8165836) WEXITSTATUS 0 Nov 05 13:03:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2481(8165836) done Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_57(8177222) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_58(8177223) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_59(8177224) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_60(8177225) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_61(8177226) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_62(8177227) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_63(8177228) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_64(8177229) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_749(7572918) WEXITSTATUS 0 Nov 05 13:03:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_749(7572918) done Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_65(8177230) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_66(8177231) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_67(8177232) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_68(8177233) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_69(8177234) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_70(8177235) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_71(8177236) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_72(8177237) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_903(7573072) WEXITSTATUS 0 Nov 05 13:03:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_903(7573072) done Nov 05 13:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2280(8165124) WEXITSTATUS 0 Nov 05 13:03:10 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2280(8165124) done Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_73(8177238) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_74(8177239) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_75(8177240) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_76(8177241) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_77(8177242) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_78(8177243) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_79(8177244) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177164_80(8177245) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:03:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_91(8171742) WEXITSTATUS 0 Nov 05 13:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_91(8171742) done Nov 05 13:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2667(8166444) WEXITSTATUS 0 Nov 05 13:03:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2667(8166444) done Nov 05 13:03:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_83(8171734) WEXITSTATUS 0 Nov 05 13:03:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_83(8171734) done Nov 05 13:03:19 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_32(8176798) WEXITSTATUS 0 Nov 05 13:03:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_32(8176798) done Nov 05 13:03:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_[81-99] uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8167851 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_0 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_2 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_4 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_6 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_8 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_3 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_5 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_7 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_1 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_9 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_10 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_11 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_12 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_13 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_14 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_15 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_16 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_17 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_18 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_19 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_20 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_21 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_22 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_23 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_24 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_27 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_25 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_32 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_28 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_26 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_31 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_29 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_33 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_30 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_34 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_35 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_36 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_38 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_37 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_39 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_40 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_41 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_43 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_44 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_42 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_45 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_48 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_47 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_49 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_46 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_50 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_53 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_51 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_56 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_54 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_58 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_59 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_52 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_55 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_61 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_60 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_62 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_57 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_63 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_64 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_65 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_66 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_67 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_68 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_69 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_70 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_71 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_72 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_73 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_74 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_75 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_76 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_77 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_78 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_79 uid 20534 Nov 05 13:03:25 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_kill_job: REQUEST_KILL_JOB JobId=8177164_80 uid 20534 Nov 05 13:03:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_51(8171702) WEXITSTATUS 0 Nov 05 13:03:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_51(8171702) done Nov 05 13:03:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5036(8158332) WEXITSTATUS 1 Nov 05 13:03:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5036(8158332) done Nov 05 13:03:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_38(8171569) WEXITSTATUS 0 Nov 05 13:03:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_38(8171569) done Nov 05 13:03:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4120(8149235) WEXITSTATUS 1 Nov 05 13:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4120(8149235) done Nov 05 13:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2491(8165881) WEXITSTATUS 0 Nov 05 13:03:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2491(8165881) done Nov 05 13:03:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_387(8172390) WEXITSTATUS 0 Nov 05 13:03:43 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_387(8172390) done Nov 05 13:03:45 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_570(8113950) has been requeued to reclaim resources for JobId=Invalid Nov 05 13:03:45 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:46 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_570(8113950) Nov 05 13:03:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_281(8172103) WEXITSTATUS 0 Nov 05 13:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_281(8172103) done Nov 05 13:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_36(8171567) WEXITSTATUS 0 Nov 05 13:03:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_36(8171567) done Nov 05 13:03:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_37(8176803) WEXITSTATUS 0 Nov 05 13:03:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_37(8176803) done Nov 05 13:04:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 13:04:04 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 13:04:12 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u11n01.grace.ycrc.yale.edu" Nov 05 13:04:12 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u11n01.grace.ycrc.yale.edu" Nov 05 13:04:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175213 WEXITSTATUS 0 Nov 05 13:04:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8175213 done Nov 05 13:04:13 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n04.grace.ycrc.yale.edu" Nov 05 13:04:13 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n04.grace.ycrc.yale.edu" Nov 05 13:04:13 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u11n02.grace.ycrc.yale.edu" Nov 05 13:04:13 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u11n02.grace.ycrc.yale.edu" Nov 05 13:04:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:14 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u06n03.grace.ycrc.yale.edu" Nov 05 13:04:14 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u06n03.grace.ycrc.yale.edu" Nov 05 13:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_49(8171700) WEXITSTATUS 0 Nov 05 13:04:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_49(8171700) done Nov 05 13:04:15 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n03.grace.ycrc.yale.edu" Nov 05 13:04:15 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n03.grace.ycrc.yale.edu" Nov 05 13:04:16 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n04.grace.ycrc.yale.edu" Nov 05 13:04:16 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n04.grace.ycrc.yale.edu" Nov 05 13:04:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:18 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n02.grace.ycrc.yale.edu" Nov 05 13:04:18 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n02.grace.ycrc.yale.edu" Nov 05 13:04:19 r912u23n01 slurmctld[163760]: slurmctld: error: _find_node_record: lookup failure for node "r804u09n01.grace.ycrc.yale.edu" Nov 05 13:04:19 r912u23n01 slurmctld[163760]: error: _find_node_record: lookup failure for node "r804u09n01.grace.ycrc.yale.edu" Nov 05 13:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7306(8165990) WEXITSTATUS 1 Nov 05 13:04:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7306(8165990) done Nov 05 13:04:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7224(8165872) WEXITSTATUS 1 Nov 05 13:04:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7224(8165872) done Nov 05 13:04:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_30(8176796) WEXITSTATUS 0 Nov 05 13:04:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_30(8176796) done Nov 05 13:04:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_482(8172538) WEXITSTATUS 0 Nov 05 13:04:35 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_482(8172538) done Nov 05 13:04:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_38(8176804) WEXITSTATUS 0 Nov 05 13:04:39 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_38(8176804) done Nov 05 13:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 WEXITSTATUS 0 Nov 05 13:04:40 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=6321884 done Nov 05 13:04:40 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6321884 Nov 05 13:04:41 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:41 r912u23n01 slurmctld[163760]: slurmctld: job_complete: invalid JobId=8166311 Nov 05 13:04:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8166311 error Invalid job id specified Nov 05 13:04:41 r912u23n01 slurmctld[163760]: slurmctld: job_complete: invalid JobId=8166311 Nov 05 13:04:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8166311 error Invalid job id specified Nov 05 13:04:41 r912u23n01 slurmctld[163760]: slurmctld: job_complete: invalid JobId=8166311 Nov 05 13:04:41 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_complete_job_allocation: JobId=8166311 error Invalid job id specified Nov 05 13:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166327 WTERMSIG 9 Nov 05 13:04:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8166327 done Nov 05 13:04:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:47 r912u23n01 slurmctld[163760]: slurmctld: error: accounts_list_build: No association for account wv48-ct-hmis Nov 05 13:04:47 r912u23n01 slurmctld[163760]: error: accounts_list_build: No association for account wv48-ct-hmis Nov 05 13:04:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_90(8171741) WEXITSTATUS 0 Nov 05 13:04:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_90(8171741) done Nov 05 13:04:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:04:58 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177246 InitPrio=27624 usec=3283 Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_0(8177247) NodeList=r907u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_1(8177248) NodeList=r907u01n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_2(8177249) NodeList=r907u01n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_3(8177250) NodeList=r907u01n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_4(8177251) NodeList=r907u01n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_5(8177252) NodeList=r907u01n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_6(8177253) NodeList=r907u01n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_7(8177254) NodeList=r907u01n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_8(8177255) NodeList=r907u01n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_9(8177256) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_10(8177257) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_11(8177258) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_12(8177259) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_13(8177260) NodeList=r907u05n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_14(8177261) NodeList=r907u05n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_15(8177262) NodeList=r907u05n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_16(8177263) NodeList=r907u05n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_17(8177264) NodeList=r907u05n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_18(8177265) NodeList=r907u05n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_19(8177266) NodeList=r907u05n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_20(8177267) NodeList=r907u05n03 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_21(8177268) NodeList=r907u05n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_22(8177269) NodeList=r907u05n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_23(8177270) NodeList=r907u05n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_24(8177271) NodeList=r907u05n04 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_25(8177272) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_26(8177273) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_27(8177274) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_28(8177275) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_29(8177276) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_30(8177277) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_31(8177278) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_32(8177279) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_33(8177280) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_34(8177281) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_35(8177282) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_36(8177283) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_37(8177284) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_38(8177285) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_39(8177286) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_40(8177287) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_41(8177288) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_42(8177289) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_43(8177290) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_44(8177291) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_45(8177292) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_46(8177293) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_47(8177294) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_48(8177295) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_49(8177296) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_50(8177297) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_51(8177298) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_52(8177299) NodeList=r816u03n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_53(8177300) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_54(8177301) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_55(8177302) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_56(8177303) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_57(8177304) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_58(8177305) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_59(8177306) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_60(8177307) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_61(8177308) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_62(8177309) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_63(8177310) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_64(8177311) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_65(8177312) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_66(8177313) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_67(8177314) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_68(8177315) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_69(8177316) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_70(8177317) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_71(8177318) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_72(8177319) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_73(8177320) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_74(8177321) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_75(8177322) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_76(8177323) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_77(8177324) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_78(8177325) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177246_79(8177246) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:04:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_70(8177326) in pi_ohern on r917u13n02 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_71(8177327) in pi_ohern on r917u13n02 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_72(8177328) in pi_ohern on r815u03n11 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_73(8177329) in pi_ohern on r815u03n11 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_74(8177330) in pi_ohern on r815u03n12 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_75(8177331) in pi_ohern on r903u24n01 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_76(8177332) in pi_ohern on r911u32n01 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_77(8177333) in pi_ohern on r911u32n01 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_78(8177334) in pi_ohern on r911u32n01 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_79(8177335) in pi_ohern on r913u32n01 Nov 05 13:05:00 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_80(8177336) in pi_ohern on r913u35n01 Nov 05 13:05:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8091(8172678) WEXITSTATUS 1 Nov 05 13:05:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8091(8172678) done Nov 05 13:05:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_35(8176801) WEXITSTATUS 0 Nov 05 13:05:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_35(8176801) done Nov 05 13:05:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4061(8177337) in gpu on r907u32n01 Nov 05 13:05:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r807u03n01 Nov 05 13:05:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3017(8177338) in day on r807u03n01 Nov 05 13:05:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3018(8177339) in day on r919u09n02 Nov 05 13:05:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3019(8177340) in day on r815u09n01 Nov 05 13:05:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3020(8177341) in day on r815u09n02 Nov 05 13:05:02 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3021(8177342) in day on r815u09n05 Nov 05 13:05:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3234(8132746) WEXITSTATUS 1 Nov 05 13:05:03 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3234(8132746) done Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_201(8177343) in day on r807u03n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_202(8177344) in day on r807u03n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_203(8177345) in day on r913u01n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_204(8177346) in day on r913u01n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_205(8177347) in day on r913u01n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_206(8177348) in day on r913u01n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_207(8177349) in day on r913u01n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_208(8177350) in day on r918u09n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_209(8177351) in day on r918u09n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_210(8177352) in day on r918u09n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_211(8177353) in day on r918u09n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_212(8177354) in day on r918u32n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_213(8177355) in day on r918u32n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_214(8177356) in day on r918u32n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_215(8177357) in day on r918u32n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_216(8177358) in day on r918u36n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_217(8177359) in day on r918u36n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_218(8177360) in day on r919u01n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_219(8177361) in day on r919u01n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_220(8177362) in day on r919u01n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_221(8177363) in day on r919u01n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_222(8177364) in day on r919u01n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_223(8177365) in day on r919u01n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_224(8177366) in day on r919u01n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_225(8177367) in day on r919u05n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_226(8177368) in day on r919u05n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_227(8177369) in day on r919u05n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_228(8177370) in day on r919u09n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_229(8177371) in day on r919u09n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_230(8177372) in day on r919u09n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_231(8177373) in day on r919u09n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_232(8177374) in day on r919u32n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_233(8177375) in day on r919u32n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_234(8177376) in day on r919u32n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_235(8177377) in day on r919u32n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_236(8177378) in day on r919u36n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_237(8177379) in day on r919u36n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_238(8177380) in day on r919u36n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_239(8177381) in day on r805u02n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_240(8177382) in day on r805u02n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_241(8177383) in day on r805u02n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_242(8177384) in day on r805u02n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_243(8177385) in day on r805u02n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_244(8177386) in day on r805u02n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_245(8177387) in day on r805u02n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_246(8177388) in day on r805u02n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_247(8177389) in day on r805u02n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_248(8177390) in day on r805u04n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_249(8177391) in day on r805u04n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_250(8177392) in day on r805u04n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_251(8177393) in day on r805u04n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_252(8177394) in day on r805u04n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_253(8177395) in day on r805u04n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_254(8177396) in day on r805u04n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_255(8177397) in day on r805u04n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_256(8177398) in day on r805u06n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_257(8177399) in day on r805u06n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_258(8177400) in day on r805u06n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_259(8177401) in day on r805u06n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_260(8177402) in day on r805u06n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_261(8177403) in day on r805u06n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_262(8177404) in day on r805u09n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_263(8177405) in day on r805u09n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_264(8177406) in day on r805u09n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_265(8177407) in day on r805u09n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_266(8177408) in day on r805u09n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_267(8177409) in day on r805u09n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_268(8177410) in day on r805u11n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_269(8177411) in day on r805u11n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_270(8177412) in day on r805u11n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_271(8177413) in day on r805u11n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_272(8177414) in day on r805u11n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_273(8177415) in day on r805u11n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_274(8177416) in day on r805u14n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_275(8177417) in day on r805u14n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_276(8177418) in day on r805u14n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_277(8177419) in day on r805u14n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_278(8177420) in day on r805u14n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_279(8177421) in day on r805u14n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_280(8177422) in day on r805u14n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_281(8177423) in day on r805u14n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_282(8177424) in day on r805u14n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_283(8177425) in day on r805u14n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_284(8177426) in day on r805u14n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_285(8177427) in day on r805u23n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_286(8177428) in day on r805u23n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_287(8177429) in day on r805u23n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_288(8177430) in day on r805u23n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_289(8177431) in day on r805u23n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_290(8177432) in day on r805u25n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_291(8177433) in day on r805u25n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_292(8177434) in day on r805u25n02 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_293(8177435) in day on r805u25n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_294(8177436) in day on r805u25n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_295(8177437) in day on r805u25n03 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_296(8177438) in day on r805u27n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_297(8177439) in day on r805u27n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_298(8177440) in day on r805u27n01 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_299(8177441) in day on r805u27n04 Nov 05 13:05:04 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_300(8177442) in day on r805u27n04 Nov 05 13:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7449(8166278) WEXITSTATUS 1 Nov 05 13:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7449(8166278) done Nov 05 13:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2367(8165574) WEXITSTATUS 0 Nov 05 13:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2367(8165574) done Nov 05 13:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5024(8158315) WEXITSTATUS 1 Nov 05 13:05:05 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5024(8158315) done Nov 05 13:05:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_903(8172184) WEXITSTATUS 0 Nov 05 13:05:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_903(8172184) done Nov 05 13:05:07 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:900 Nov 05 13:05:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_394(8172397) WEXITSTATUS 0 Nov 05 13:05:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_394(8172397) done Nov 05 13:05:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5943(8162938) in scavenge on r802u11n03 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5784(8162734) in scavenge on r815u09n10 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5785(8162735) in scavenge on r815u15n11 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5828(8162790) in scavenge on r816u29n05 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5285(8158733) in scavenge on r816u35n05 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5712(8162425) in scavenge on r816u15n03 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_2948(8128632) in scavenge on r816u15n04 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4776(8157458) in scavenge on r816u15n07 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5551(8161041) in scavenge on r816u15n07 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5722(8162524) in scavenge on r815u03n05 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7572167_1000(7573169) in scavenge on r816u09n10 Nov 05 13:05:10 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_338(8114605) in scavenge on r815u03n05 Nov 05 13:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_3(8177443) in scavenge on r801u30n01 Nov 05 13:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_4(8177444) in scavenge on r908u35n02 Nov 05 13:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_5(8177445) in scavenge on r908u35n03 Nov 05 13:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_6(8177446) in scavenge on r908u35n04 Nov 05 13:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_7(8177447) in scavenge on r802u06n01 Nov 05 13:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_8(8177448) in scavenge on r802u06n01 Nov 05 13:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_9(8177449) in scavenge on r802u06n01 Nov 05 13:05:12 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177033_10(8177033) in scavenge on r802u06n01 Nov 05 13:05:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_548(8172634) WEXITSTATUS 0 Nov 05 13:05:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_548(8172634) done Nov 05 13:05:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_277(8172099) WEXITSTATUS 0 Nov 05 13:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_277(8172099) done Nov 05 13:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_24(8171555) WEXITSTATUS 0 Nov 05 13:05:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_24(8171555) done Nov 05 13:05:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_25(8171556) WEXITSTATUS 0 Nov 05 13:05:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_25(8171556) done Nov 05 13:05:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_39(8171570) WEXITSTATUS 0 Nov 05 13:05:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_39(8171570) done Nov 05 13:05:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4061(8177337) WEXITSTATUS 0 Nov 05 13:05:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4061(8177337) done Nov 05 13:05:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_388(8172391) WEXITSTATUS 0 Nov 05 13:05:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_388(8172391) done Nov 05 13:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_48(8171699) WEXITSTATUS 0 Nov 05 13:05:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_48(8171699) done Nov 05 13:05:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_897(8172178) WEXITSTATUS 0 Nov 05 13:05:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8170528_897(8172178) done Nov 05 13:05:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177450 InitPrio=33576 usec=5132 Nov 05 13:05:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177451 InitPrio=33576 usec=2101 Nov 05 13:05:30 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177452 InitPrio=33576 usec=2380 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177453 InitPrio=33576 usec=2156 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177454 InitPrio=33576 usec=1323 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177455 InitPrio=33576 usec=1470 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177456 InitPrio=33576 usec=1593 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177457 InitPrio=33576 usec=1612 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177458 InitPrio=33576 usec=1106 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177459 InitPrio=33576 usec=1286 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177460 InitPrio=33576 usec=1246 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177461 InitPrio=33576 usec=985 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177462 InitPrio=33576 usec=1012 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177463 InitPrio=33576 usec=947 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177464 InitPrio=33576 usec=981 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177465 InitPrio=33576 usec=936 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177466 InitPrio=33576 usec=928 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177467 InitPrio=33576 usec=1246 Nov 05 13:05:31 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177468 InitPrio=33576 usec=1042 Nov 05 13:05:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_284(8172106) WEXITSTATUS 0 Nov 05 13:05:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_284(8172106) done Nov 05 13:05:37 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154718 WEXITSTATUS 0 Nov 05 13:05:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8154718 done Nov 05 13:05:40 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:45 r912u23n01 slurmctld[163760]: slurmctld: Time limit exhausted for JobId=8166147 Nov 05 13:05:47 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_41(8176807) WEXITSTATUS 0 Nov 05 13:05:49 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_41(8176807) done Nov 05 13:05:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:05:57 r912u23n01 slurmctld[163760]: slurmctld: node r804u11n01 returned to service Nov 05 13:05:58 r912u23n01 slurmctld[163760]: slurmctld: node r804u11n02 returned to service Nov 05 13:05:58 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n04 returned to service Nov 05 13:05:58 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n03 returned to service Nov 05 13:05:58 r912u23n01 slurmctld[163760]: slurmctld: node r804u06n03 returned to service Nov 05 13:05:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:01 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n04 returned to service Nov 05 13:06:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_363(8172264) WEXITSTATUS 0 Nov 05 13:06:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_363(8172264) done Nov 05 13:06:03 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n02 returned to service Nov 05 13:06:04 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:04 r912u23n01 slurmctld[163760]: slurmctld: node r804u09n01 returned to service Nov 05 13:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7779(8169118) WEXITSTATUS 1 Nov 05 13:06:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7779(8169118) done Nov 05 13:06:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_261(8172083) WEXITSTATUS 0 Nov 05 13:06:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_261(8172083) done Nov 05 13:06:11 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 13:06:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 13:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_349(8172250) WEXITSTATUS 0 Nov 05 13:06:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_349(8172250) done Nov 05 13:06:15 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_342(8172243) WEXITSTATUS 0 Nov 05 13:06:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_342(8172243) done Nov 05 13:06:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=6327612 Nov 05 13:06:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4059(8176916) WEXITSTATUS 0 Nov 05 13:06:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4059(8176916) done Nov 05 13:06:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177469 InitPrio=27638 usec=5058 Nov 05 13:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_432(8172459) WEXITSTATUS 0 Nov 05 13:06:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_432(8172459) done Nov 05 13:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_433(8172460) WEXITSTATUS 0 Nov 05 13:06:23 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_433(8172460) done Nov 05 13:06:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_0(8177470) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_1(8177471) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:24 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_2(8177472) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:24 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1000(7573169) WEXITSTATUS 0 Nov 05 13:06:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7572167_1000(7573169) done Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_3(8177473) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_4(8177474) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_5(8177475) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_6(8177476) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_7(8177477) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_8(8177478) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_9(8177479) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_10(8177480) NodeList=r816u09n10 #CPUs=1 Partition=pi_fedorov Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_97(8171748) WEXITSTATUS 0 Nov 05 13:06:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_97(8171748) done Nov 05 13:06:30 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2462(8165763) WEXITSTATUS 0 Nov 05 13:06:32 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2462(8165763) done Nov 05 13:06:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_226(8172048) WEXITSTATUS 0 Nov 05 13:06:33 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_226(8172048) done Nov 05 13:06:36 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8180(8174119) WEXITSTATUS 1 Nov 05 13:06:36 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8180(8174119) done Nov 05 13:06:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_45(8171696) WEXITSTATUS 0 Nov 05 13:06:38 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_45(8171696) done Nov 05 13:06:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_486(8172542) WEXITSTATUS 0 Nov 05 13:06:48 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_486(8172542) done Nov 05 13:06:49 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_7(8165334) WEXITSTATUS 0 Nov 05 13:06:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8165262_7(8165334) done Nov 05 13:06:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_280(8172102) WEXITSTATUS 0 Nov 05 13:06:53 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_280(8172102) done Nov 05 13:06:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7942(8170525) WEXITSTATUS 1 Nov 05 13:06:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7942(8170525) done Nov 05 13:07:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2299(8165185) WEXITSTATUS 0 Nov 05 13:07:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2299(8165185) done Nov 05 13:07:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7231(8165884) WEXITSTATUS 1 Nov 05 13:07:02 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7231(8165884) done Nov 05 13:07:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_93(8171744) WEXITSTATUS 0 Nov 05 13:07:06 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_93(8171744) done Nov 05 13:07:07 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_283(8172105) WEXITSTATUS 0 Nov 05 13:07:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_283(8172105) done Nov 05 13:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2615(8166253) WEXITSTATUS 0 Nov 05 13:07:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2615(8166253) done Nov 05 13:07:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:16 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_721(8114202) has been requeued to reclaim resources for JobId=Invalid Nov 05 13:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2349(8165470) WEXITSTATUS 0 Nov 05 13:07:17 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2349(8165470) done Nov 05 13:07:17 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_721(8114202) Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_11(8177481) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_12(8177482) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_13(8177483) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_14(8177484) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_15(8177485) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_16(8177486) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_17(8177487) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_18(8177488) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_19(8177489) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_20(8177490) NodeList=r906u01n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_437(8114704) WEXITSTATUS 0 Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_437(8114704) done Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_547(8172633) WEXITSTATUS 0 Nov 05 13:07:18 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_547(8172633) done Nov 05 13:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_47(8171698) WEXITSTATUS 0 Nov 05 13:07:19 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_47(8171698) done Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_21(8177491) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_22(8177492) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_23(8177493) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_24(8177494) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_25(8177495) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_26(8177496) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_27(8177497) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_28(8177498) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_29(8177499) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_30(8177500) NodeList=r816u03n12 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_450(8114717) WEXITSTATUS 0 Nov 05 13:07:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8114217_450(8114717) done Nov 05 13:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2333(8165367) WEXITSTATUS 0 Nov 05 13:07:21 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2333(8165367) done Nov 05 13:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172976 WEXITSTATUS 0 Nov 05 13:07:22 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8172976 done Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_31(8177501) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_32(8177502) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_33(8177503) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_34(8177504) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_35(8177505) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_36(8177506) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_37(8177507) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_38(8177508) NodeList=r816u09n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:07:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3955(8169336) WEXITSTATUS 0 Nov 05 13:07:25 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3955(8169336) done Nov 05 13:07:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3612(8136977) WEXITSTATUS 1 Nov 05 13:07:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_3612(8136977) done Nov 05 13:07:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7729(8168720) WEXITSTATUS 1 Nov 05 13:07:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_7729(8168720) done Nov 05 13:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2255(8165067) WEXITSTATUS 0 Nov 05 13:07:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2255(8165067) done Nov 05 13:07:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_102(8171753) WEXITSTATUS 0 Nov 05 13:07:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_102(8171753) done Nov 05 13:07:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_50(8171701) WEXITSTATUS 0 Nov 05 13:07:34 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_50(8171701) done Nov 05 13:07:35 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_391(8172394) WEXITSTATUS 0 Nov 05 13:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_391(8172394) done Nov 05 13:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2282(8165131) WEXITSTATUS 0 Nov 05 13:07:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2282(8165131) done Nov 05 13:07:42 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2362(8165555) WEXITSTATUS 0 Nov 05 13:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2362(8165555) done Nov 05 13:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_274(8172096) WEXITSTATUS 0 Nov 05 13:07:42 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_274(8172096) done Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_81(8177509) in pi_ohern on r917u13n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_82(8177510) in pi_ohern on r917u13n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_83(8177511) in pi_ohern on r917u13n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_84(8177512) in pi_ohern on r917u13n02 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_85(8177513) in pi_ohern on r917u13n04 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_86(8177514) in pi_ohern on r815u03n11 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_87(8177515) in pi_ohern on r815u03n11 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_88(8177516) in pi_ohern on r815u03n11 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_89(8177517) in pi_ohern on r815u03n12 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_90(8177518) in pi_ohern on r815u03n12 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_91(8177519) in pi_ohern on r815u03n12 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_92(8177520) in pi_ohern on r815u03n12 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_93(8177521) in pi_ohern on r815u03n12 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_94(8177522) in pi_ohern on r815u03n12 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_95(8177523) in pi_ohern on r815u03n12 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_96(8177524) in pi_ohern on r815u03n12 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_97(8177525) in pi_ohern on r807u22n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_98(8177526) in pi_ohern on r807u22n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_99(8177527) in pi_ohern on r807u22n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_100(8177528) in pi_ohern on r913u32n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_101(8177529) in pi_ohern on r913u32n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_102(8177530) in pi_ohern on r913u35n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_103(8177531) in pi_ohern on r913u35n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_104(8177532) in pi_ohern on r903u12n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_105(8177533) in pi_ohern on r903u24n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_106(8177534) in pi_ohern on r903u24n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_107(8177535) in pi_ohern on r903u24n01 Nov 05 13:07:43 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_108(8177536) in pi_ohern on r903u26n01 Nov 05 13:07:44 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8173029 in day on r815u09n08 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4062(8177537) in gpu on r808u08n01 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4063(8177538) in gpu on r4513u26n01 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4064(8177539) in gpu on r907u32n01 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3022(8177540) in day on r807u11n04 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3023(8177541) in day on r906u41n03 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3024(8177542) in day on r909u13n04 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3025(8177543) in day on r805u04n01 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3026(8177544) in day on r805u11n01 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3027(8177545) in day on r805u27n04 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3028(8177546) in day on r815u03n08 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3029(8177547) in day on r815u09n03 Nov 05 13:07:45 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3030(8177548) in day on r815u15n05 Nov 05 13:07:46 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1000 Nov 05 13:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135448 WEXITSTATUS 0 Nov 05 13:07:46 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8135448 done Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_301(8177549) in day on r906u41n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_302(8177550) in day on r906u41n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_303(8177551) in day on r805u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_304(8177552) in day on r805u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_305(8177553) in day on r805u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_306(8177554) in day on r805u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_307(8177555) in day on r805u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_308(8177556) in day on r805u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_309(8177557) in day on r805u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_310(8177558) in day on r805u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_311(8177559) in day on r805u14n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_312(8177560) in day on r805u14n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_313(8177561) in day on r805u27n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_314(8177562) in day on r918u36n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_315(8177563) in day on r919u01n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_316(8177564) in day on r919u05n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_317(8177565) in day on r806u02n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_318(8177566) in day on r806u02n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_319(8177567) in day on r806u02n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_320(8177568) in day on r806u02n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_321(8177569) in day on r806u02n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_322(8177570) in day on r806u02n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_323(8177571) in day on r806u02n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_324(8177572) in day on r806u02n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_325(8177573) in day on r806u02n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_326(8177574) in day on r806u02n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_327(8177575) in day on r806u02n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_328(8177576) in day on r806u02n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_329(8177577) in day on r806u02n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_330(8177578) in day on r806u02n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_331(8177579) in day on r806u04n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_332(8177580) in day on r806u04n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_333(8177581) in day on r806u04n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_334(8177582) in day on r806u04n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_335(8177583) in day on r806u04n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_336(8177584) in day on r806u04n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_337(8177585) in day on r806u04n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_338(8177586) in day on r806u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_339(8177587) in day on r806u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_340(8177588) in day on r806u06n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_341(8177589) in day on r806u06n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_342(8177590) in day on r806u06n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_343(8177591) in day on r806u06n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_344(8177592) in day on r806u06n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_345(8177593) in day on r806u06n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_346(8177594) in day on r806u06n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_347(8177595) in day on r806u06n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_348(8177596) in day on r806u06n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_349(8177597) in day on r806u06n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_350(8177598) in day on r806u06n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_351(8177599) in day on r806u06n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_352(8177600) in day on r806u09n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_353(8177601) in day on r806u09n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_354(8177602) in day on r806u09n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_355(8177603) in day on r806u09n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_356(8177604) in day on r806u09n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_357(8177605) in day on r806u09n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_358(8177606) in day on r806u09n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_359(8177607) in day on r806u11n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_360(8177608) in day on r806u11n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_361(8177609) in day on r806u11n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_362(8177610) in day on r806u11n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_363(8177611) in day on r806u11n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_364(8177612) in day on r806u11n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_365(8177613) in day on r806u11n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_366(8177614) in day on r806u11n03 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_367(8177615) in day on r806u11n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_368(8177616) in day on r806u11n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_369(8177617) in day on r806u11n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_370(8177618) in day on r806u11n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_371(8177619) in day on r806u11n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_372(8177620) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_373(8177621) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_374(8177622) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_375(8177623) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_376(8177624) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_377(8177625) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_378(8177626) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_379(8177627) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_380(8177628) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_381(8177629) in day on r806u14n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_382(8177630) in day on r806u14n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_383(8177631) in day on r806u14n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_384(8177632) in day on r806u14n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_385(8177633) in day on r806u14n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_386(8177634) in day on r806u14n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_387(8177635) in day on r806u14n04 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_388(8177636) in day on r806u23n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_389(8177637) in day on r806u23n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_390(8177638) in day on r806u23n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_391(8177639) in day on r806u23n02 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_392(8177640) in day on r816u03n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_393(8177641) in day on r816u03n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_394(8177642) in day on r816u03n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_395(8177643) in day on r816u03n01 Nov 05 13:07:47 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8177450 in scavenge on r815u09n09 Nov 05 13:07:48 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u13n03 Nov 05 13:07:48 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u24n03 Nov 05 13:07:48 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u24n02 Nov 05 13:07:48 r912u23n01 slurmctld[163760]: slurmctld: reboot request queued for nodes r905u13n04 Nov 05 13:07:48 r912u23n01 slurmctld[163760]: slurmctld: retry_list retry_list_size:192 msg_type=REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG,REQUEST_BATCH_JOB_LAUNCH,REQUEST_LAUNCH_PROLOG Nov 05 13:07:50 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5077(8158389) WEXITSTATUS 1 Nov 05 13:07:50 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_5077(8158389) done Nov 05 13:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4062(8177537) WEXITSTATUS 0 Nov 05 13:07:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4062(8177537) done Nov 05 13:07:54 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4725(8156736) in scavenge on r908u35n01 Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4754(8157104) in scavenge on r908u35n01 Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5838(8162805) in scavenge on r801u09n04 Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4537(8155942) in scavenge on r802u11n01 Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4542(8155947) in scavenge on r816u15n10 Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_4952(8158197) in scavenge on r906u39n01 Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5895(8162871) in scavenge on r815u03n05 Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8112304_5839(8162806) in scavenge on r815u09n01 Nov 05 13:07:57 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8114217_427(8114694) in scavenge on r815u09n09 Nov 05 13:07:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4037(8176632) WEXITSTATUS 0 Nov 05 13:07:59 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4037(8176632) done Nov 05 13:08:00 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_556(8172642) WEXITSTATUS 0 Nov 05 13:08:00 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_556(8172642) done Nov 05 13:08:03 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_4(8171535) WEXITSTATUS 0 Nov 05 13:08:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_4(8171535) done Nov 05 13:08:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_351(8172252) WEXITSTATUS 0 Nov 05 13:08:13 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_351(8172252) done Nov 05 13:08:14 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_1929(8125286) has been requeued to reclaim resources for JobId=8177469_39(8177469) Nov 05 13:08:14 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_286(8172108) WEXITSTATUS 0 Nov 05 13:08:14 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_286(8172108) done Nov 05 13:08:16 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_1929(8125286) Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_39(8177644) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_40(8177645) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_41(8177646) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_42(8177647) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_43(8177648) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_44(8177649) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_45(8177650) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_46(8177651) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_47(8177652) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_48(8177653) NodeList=r907u09n02 #CPUs=1 Partition=pi_fedorov Nov 05 13:08:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:22 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177654 InitPrio=34236 usec=3765 Nov 05 13:08:23 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_121(8171891) WEXITSTATUS 0 Nov 05 13:08:24 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_121(8171891) done Nov 05 13:08:26 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_271(8172093) WEXITSTATUS 0 Nov 05 13:08:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_271(8172093) done Nov 05 13:08:32 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:52 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177655 InitPrio=27638 usec=3506 Nov 05 13:08:53 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:08:56 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177654 NodeList=r909u29n02 #CPUs=4 Partition=day Nov 05 13:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_392(8172395) WEXITSTATUS 0 Nov 05 13:08:58 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_392(8172395) done Nov 05 13:08:59 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_386(8172389) WEXITSTATUS 0 Nov 05 13:09:04 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_386(8172389) done Nov 05 13:09:05 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_549(8172635) WEXITSTATUS 0 Nov 05 13:09:08 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_549(8172635) done Nov 05 13:09:09 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3064(8129875) has been requeued to reclaim resources for JobId=8177469_49(8177469) Nov 05 13:09:09 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:11 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3064(8129875) Nov 05 13:09:11 r912u23n01 slurmctld[163760]: slurmctld: _slurm_rpc_submit_batch_job: JobId=8177656 InitPrio=27412 usec=8076 Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_49(8177657) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_50(8177658) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_51(8177659) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_52(8177660) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_53(8177661) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_54(8177662) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_55(8177663) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_56(8177664) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_57(8177665) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_58(8177666) NodeList=r816u03n09 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:12 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3981(8170640) WEXITSTATUS 0 Nov 05 13:09:16 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_3981(8170640) done Nov 05 13:09:17 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2325(8165314) WEXITSTATUS 0 Nov 05 13:09:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2325(8165314) done Nov 05 13:09:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_352(8172253) WEXITSTATUS 0 Nov 05 13:09:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_352(8172253) done Nov 05 13:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4709(8156592) WEXITSTATUS 1 Nov 05 13:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_4709(8156592) done Nov 05 13:09:28 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_282(8172104) WEXITSTATUS 0 Nov 05 13:09:28 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_282(8172104) done Nov 05 13:09:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_360(8172261) WEXITSTATUS 0 Nov 05 13:09:29 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_360(8172261) done Nov 05 13:09:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2533(8166078) WEXITSTATUS 0 Nov 05 13:09:30 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2533(8166078) done Nov 05 13:09:31 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_269(8172091) WEXITSTATUS 0 Nov 05 13:09:31 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_269(8172091) done Nov 05 13:09:34 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2550(8166099) WEXITSTATUS 0 Nov 05 13:09:37 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2550(8166099) done Nov 05 13:09:38 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_1(8176754) WEXITSTATUS 0 Nov 05 13:09:41 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8176752_1(8176754) done Nov 05 13:09:43 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:47 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3618(8136983) has been requeued to reclaim resources for JobId=Invalid Nov 05 13:09:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_395(8172398) WEXITSTATUS 0 Nov 05 13:09:47 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_395(8172398) done Nov 05 13:09:48 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:48 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3618(8136983) Nov 05 13:09:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_276(8172098) WEXITSTATUS 0 Nov 05 13:09:51 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_276(8172098) done Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_59(8177667) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_60(8177668) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_61(8177669) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_62(8177670) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_63(8177671) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_64(8177672) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_65(8177673) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_66(8177674) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_67(8177675) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_68(8177676) NodeList=r907u05n01 #CPUs=1 Partition=pi_fedorov Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_122(8171892) WEXITSTATUS 0 Nov 05 13:09:52 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_122(8171892) done Nov 05 13:09:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4064(8177539) WEXITSTATUS 0 Nov 05 13:09:54 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4064(8177539) done Nov 05 13:09:55 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2346(8165467) WEXITSTATUS 0 Nov 05 13:09:55 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8123792_2346(8165467) done Nov 05 13:09:56 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_63(8171714) WEXITSTATUS 0 Nov 05 13:10:01 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_63(8171714) done Nov 05 13:10:02 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_266(8172088) WEXITSTATUS 0 Nov 05 13:10:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_266(8172088) done Nov 05 13:10:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4063(8177538) WEXITSTATUS 0 Nov 05 13:10:07 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=7907002_4063(8177538) done Nov 05 13:10:08 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8157(8172962) WEXITSTATUS 1 Nov 05 13:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8112304_8157(8172962) done Nov 05 13:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_3(8171534) WEXITSTATUS 0 Nov 05 13:10:09 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_3(8171534) done Nov 05 13:10:10 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=1664178 Nodelist=r802u14n04 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184127 Nodelist=r906u37n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184137 Nodelist=r801u11n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184139 Nodelist=r801u32n04 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184141 Nodelist=r802u25n02 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=3184149 Nodelist=r914u13n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158601 Nodelist=r801u25n03 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158717 Nodelist=r908u28n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4158783 Nodelist=r915u32n02 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465358 Nodelist=r802u27n02 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465365 Nodelist=r913u05n02 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4465381 Nodelist=r801u06n04,r801u14n04 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507122 Nodelist=r802u23n03 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4507144 Nodelist=r904u10n04 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557677 Nodelist=r801u09n02 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=4557679 Nodelist=r912u01n03 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=5965779 Nodelist=r905u24n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619773 Nodelist=r801u27n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619806 Nodelist=r801u11n03 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619808 Nodelist=r914u09n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619818 Nodelist=r801u02n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619819 Nodelist=r801u02n02 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7619836 Nodelist=r801u27n04 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903368 Nodelist=r915u32n03 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903387 Nodelist=r801u23n04,r801u32n01 Nov 05 13:10:12 r912u23n01 slurmctld[163760]: slurmctld: Resending TERMINATE_JOB request JobId=7903402 Nodelist=r912u05n02 Nov 05 13:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_362(8172263) WEXITSTATUS 0 Nov 05 13:10:15 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_362(8172263) done Nov 05 13:10:16 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_16(8171547) WEXITSTATUS 0 Nov 05 13:10:20 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_16(8171547) done Nov 05 13:10:21 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_98(8171749) WEXITSTATUS 0 Nov 05 13:10:26 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_98(8171749) done Nov 05 13:10:27 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_61(8171712) WEXITSTATUS 0 Nov 05 13:10:27 r912u23n01 slurmctld[163760]: slurmctld: _job_complete: JobId=8169127_61(8171712) done Nov 05 13:10:27 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 13:10:28 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 13:10:29 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:29 r912u23n01 slurmctld[163760]: slurmctld: accounting_storage/slurmdbd: _agent: agent_count:1300 Nov 05 13:10:29 r912u23n01 slurmctld[163760]: slurmctld: preempted JobId=8112304_3756(8142530) has been requeued to reclaim resources for JobId=8177655_0(8177655) Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_109(8177677) in pi_ohern on r917u13n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_110(8177678) in pi_ohern on r917u13n02 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_111(8177679) in pi_ohern on r917u13n03 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_112(8177680) in pi_ohern on r917u13n03 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_113(8177681) in pi_ohern on r917u29n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_114(8177682) in pi_ohern on r917u29n02 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_115(8177683) in pi_ohern on r917u29n04 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_116(8177684) in pi_ohern on r815u03n11 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_117(8177685) in pi_ohern on r815u03n12 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_118(8177686) in pi_ohern on r815u03n12 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_119(8177687) in pi_ohern on r815u03n12 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_120(8177688) in pi_ohern on r815u03n12 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_121(8177689) in pi_ohern on r911u32n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_122(8177690) in pi_ohern on r807u22n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_123(8177691) in pi_ohern on r807u24n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_124(8177692) in pi_ohern on r807u24n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_125(8177693) in pi_ohern on r807u24n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_126(8177694) in pi_ohern on r807u24n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_127(8177695) in pi_ohern on r903u12n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_128(8177696) in pi_ohern on r903u24n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_129(8177697) in pi_ohern on r903u24n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_130(8177698) in pi_ohern on r913u32n01 Nov 05 13:10:30 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8169130_131(8177699) in pi_ohern on r913u35n01 Nov 05 13:10:31 r912u23n01 slurmctld[163760]: slurmctld: Requeuing JobId=8112304_3756(8142530) Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4065(8177700) in gpu on r903u34n01 Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4066(8177701) in gpu on r808u08n01 Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4067(8177702) in gpu on r808u14n01 Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4068(8177703) in gpu on r4513u26n01 Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=7907002_4069(8177704) in gpu on r907u32n01 Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=6327612 in day on r914u05n04 Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3031(8177705) in day on r807u11n03 Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3032(8177706) in day on r918u01n02 Nov 05 13:10:32 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8123792_3033(8177707) in day on r815u03n06 Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_69(8177708) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_70(8177709) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_71(8177710) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_72(8177711) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_73(8177712) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_74(8177713) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_75(8177714) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_76(8177715) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_77(8177716) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched: Allocate JobId=8177469_78(8177717) NodeList=r816u03n11 #CPUs=1 Partition=pi_fedorov Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: _build_node_list: No nodes satisfy JobId=7907002_* requirements in partition pi_ohern Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_396(8177718) in day on r909u13n04 Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_397(8177719) in day on r913u13n03 Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_398(8177720) in day on r816u03n03 Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_399(8177721) in day on r816u03n03 Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_400(8177722) in day on r816u03n03 Nov 05 13:10:33 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_401(8177723) in day on r816u03n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_402(8177724) in day on r816u03n06 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_403(8177725) in day on r816u03n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_404(8177726) in day on r816u03n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_405(8177727) in day on r816u03n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_406(8177728) in day on r816u03n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_407(8177729) in day on r816u03n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_408(8177730) in day on r816u03n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_409(8177731) in day on r816u09n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_410(8177732) in day on r816u09n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_411(8177733) in day on r816u09n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_412(8177734) in day on r816u09n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_413(8177735) in day on r816u09n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_414(8177736) in day on r816u09n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_415(8177737) in day on r816u09n05 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_416(8177738) in day on r816u09n06 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_417(8177739) in day on r816u09n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_418(8177740) in day on r816u09n08 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_419(8177741) in day on r816u09n08 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_420(8177742) in day on r816u15n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_421(8177743) in day on r816u15n04 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_422(8177744) in day on r816u15n04 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_423(8177745) in day on r816u15n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_424(8177746) in day on r816u15n07 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_425(8177747) in day on r815u23n01 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_426(8177748) in day on r815u23n01 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_427(8177749) in day on r815u23n01 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_428(8177750) in day on r815u23n03 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_429(8177751) in day on r815u23n05 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_430(8177752) in day on r815u23n05 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_431(8177753) in day on r815u23n06 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8176902_432(8177754) in day on r815u29n01 Nov 05 13:10:34 r912u23n01 slurmctld[163760]: slurmctld: sched/backfill: _start_job: Started JobId=8